How the Japanese IT Industry Destroys Talent

Part II: My struggle at the frontline of Japanese Enterprise IT

(Published in Magazine No 71: Part I: How Japan’s enterprise IT has failed to learn from its most competitive industry–manufacturing.)

By James Mok

While troops of IT project managers and engineers are being sent to ‘Death March’—a popular term to describe the brutal working conditions of extreme overtime work, tremendous customer pressure and poor office environments at the frontline of failing projects—the Japanese enterprise IT industry has been failing to cultivate innovation, reward its workers or attract young graduates. In fact, as Nikkei Computer revealed in a recent survey, more than 70% of the IT workers who responded felt that the industry is not attractive to the younger generation, who now recognize IT work as the equivalent of ‘3K’: Kitsui (hard work), Kaerenai (long hours), and Kyuryohikui (low pay). According to the Information Services 2006 white paper released by the government, talent shortage is the top issue facing the IT industry. Measures such as ITSS (IT Skill Standardization) and the strengthening of IT education are being taken. Unfortunately, these measures will be rendered futile unless deep-rooted issues concerning the very structure of the industry are tackled as well.

Absence of Takumi

During my years working as a quality inspector for many factories throughout Japan, I had the opportunity to meet many engineers and technicians who were known as ‘Takumi,’ or ‘The Master’ by their peers. In a magnetic disk factory, for instance, there was a man who could read micron level scratches with his naked eyes in many cases better than a million dollar microscope. In a metal-forming factory, I met a technician who could tell the curvatures of the metal surface just by touching it. His accuracy was comparable to that of a 3D precision measurement machine. He could then make fine adjustments on the forming machine, which was controlled by hundreds of parameters according to his feel. The highly skilled workforce led by such Takumi, with their strong tacit knowledge, constitutes a core part of Japan’s competitiveness in manufacturing.

When I switched my career to the enterprise IT industry, I was amazed by the almost non-existence of such master level engineers. In fact there are many prominent characters that have Japanese origins but most of them have developed their skills outside of Japan.

The comparison in Part I of this story highlighted inefficiencies within the enterprise IT industry, when lined up against the manufacturing industry. For instance, there’s a lack of efficient process management from services vendors, an almost non-existence of Japanese software packages entering the global market, and very few contributions from academia, in either research or application, in the field of software engineering. This is a huge contrast to the manufacturing industry, in which Japan has held the leadership position for decades. Some have suggested a possible reason for this is that Japanese people are not good at theoretical thinking. Others have suggested that the software world is English-centric, resulting in a natural disadvantage for Japanese. However, many Japanese have made important contributions in mathematics and physics, both of which require strong theoretical thinking. Taiwan and Korea have both made significant advances in the application of IT and software, despite their obvious disadvantage as non-native Englishspeaking countries. The ‘natural disadvantage’ theory, then, is merely an excuse. The real issue lies in how the Japanese IT industry is structured.

Problematic structure of the IT industry

Fundamentally speaking, there are three main, interconnected ‘spirals’ that underpin the IT industry in Japan and are largely responsible for creating the environment in which so few talented people can flourish. They are:

The interconnected ‘spirals’ that underpin the IT industry -- Click on image for enlarged view.

Let’s take a closer look at each of them.

1. The academic and industrial spiral
In the manufacturing world, tacit knowledge developed through years of experience by Takumi engineers can serve as an important competitive advantage. In the IT world, articulated knowledge that can be conveyed to others is more important. With advancements in the internet and computers, such articulated knowledge is becoming a commodity, to which anyone who makes the effort can access. Merely learning a programming language or becoming skillful in a particular IT tool can no longer lead to the path of Takumi. Instead, it requires discovery and invention of new frontiers.

In order to discover and invent new knowledge, the engineers need a solid understanding of basic theories, as well as having the capability to hypothesize and imagine. Unfortunately, a high percentage of the IT workforce in Japan has not had any formal IT or computer science education. The majority become engineers through on-job training that emphasizes memorizing the use of IT tools, a process that can hardly develop leading engineers to invent and broadcast new knowledge.

When I was an engineering student back at Stanford, most of the engineering classes were broadcast live to corporations around the world. Engineers working in the frontline participated through video conferencing together with university students. In this way, frontline workers can learn about the latest advance in academic theories and students can learn about practical applications through live discussions.

The computer facilities in the university are always supplied and upgraded by corporations. Even students of unrelated fields can get easy access to the latest computing equipment. Corporations believe that once engineers have experienced the latest tools, they will be reluctant to go back to an older version, and as a result, will actively promote the use of new tools when they enter the workplace.

This kind of intimate relationship between academia and industry is still rare in Japan.

On the other hand, corporations are not expecting new graduates to make any immediate contribution to their business. Japanese companies typically make a huge effort to re-educate new employees. There is no system in place at most corporations to evaluate and reward university graduates on what they have learnt in school, or to encourage them to apply their knowledge to their business. In this way, the first vicious circle between academic and industry is established.

This first spiral relationship results in a significant gap between academic research and industrial application. It can be thought of as a fundamental contributor to the current low-level application of software engineering principles, as well as conceptual architecture skills to IT projects in Japan.

2. Software vendors and user corporations spiral
Generally speaking, US IT managers place a strong emphasis on the importance of system architecture when deploying business solutions. In turn, software vendors utilize the latest architecture concept to constantly enhance their product. Implementation services vendors also conduct project management that reflects the benefits that were brought about by the latest architecture.

In part, the attention to system architecture by western organizations is owed to the relatively high turnover of the IT workforce. Managers need to have a firm grasp on the architecture in order to keep the system going forward.

Most Japanese IT projects tend to do a lot of custom development, specialized for each customer. The use of open, extensible and common components under a well-designed architecture is appreciated to a much less extent. Take for example the recent advancement in SOA (service oriented architecture). A recent survey has indicated that more than 60% of US corporations have strategies to adopt SOA, while less than 10% of Japanese corporations have a clear strategy. This clearly reflects a lack of conceptual skill and the limitations of the bottom-up organizational structure of Japanese corporations.

Instead of adopting an IT architectural based strategy, most Japanese user corporations are very particular about usability and the cosmetic details of screen design. Such demands do not encourage IT vendors to pay attention to the overall system architecture, or to reuse existing package software to effectively assemble business solutions. Instead, most vendors are more inclined to develop their own custom solutions from scratch in order to cope with detailed requirements, regardless of whether those requirements contribute to differentiatiation of the users’ business.

As long as the user corporations pay little attention to system architecture, Japanese software companies are not encouraged to develop applications that can meet the needs of the global market. In this way the second spiral relationship between software vendors and user corporations is formulated.

Japanese-made package software tends to suffer from the same architectural weaknesses. Custom development occupies the majority of implementation work. Hence engineers who are specialized to service a particular customer are required to support and maintain the system after cut-over. And, as such, face great difficulties when it is time to upgrade to new infrastructure technologies.

The ‘body shop’ strategy that destroys engineers

3. SI services vendors and user corporations spiral
Japanese SI vendors tend to pay less attention to combining existing solutions into design solutions of ‘clever’ architecture, instead focusing on sending troops that are distinguished by the mere number of head-counts to projects. SI business is in fact operating by ‘selling bodies.’ Violation of the Brook’s law is rampant all over the industry. (Fred Brooks published his experiences in software development some 30 years ago, describing the most common management mistake: “Adding manpower to a late software project makes it later.”)

As long as ‘clever’ architecture is not well evaluated by customers, the ‘manmonth’ based business model exerts cost pressure on the SI vendors to sell bodies at a lower cost. This business model allows no explicit merits to reward engineers or to encourage them to polish their skills.

In order to reduce costs, SI vendors are under pressure to sub-contract. It is not uncommon that a project led by a top vendor in the industry is actually conducted by many layers of subcontractors with very little involvement from the prime vendor.

To make such projects successful, project management skills are the key. However, the average skill level of project managers in Japan is generally lower than their western counterparts (see Figure 1).

Figure 1, reference IBM Japan, Ltd. (2001)

In most projects that I have known, the project managers spend all day long on the phone. Japanese project managers tend to focus solely on people management and give less importance to scientific project management methods.

Instead of being evaluated on their capability to manage the overall system architecture, Japanese IT project managers are often assessed on how they can personally relate to the team members. Taking team members out for a drink, listening to their personal issues, serving as both counselor and cheerleader, are important to strengthen a project manager’s people network. When troubles occur during the project, the project manager uses his networking skills to motivate his troops. The ability to send more people to a troubled project and request members to work longer hours to resolve troubles without complaint is generally expected and evaluated by the customers.

Unfortunately, such fire-fighting acts do not usually lead to any resolution of the root cause. Many system troubles are caused by obsolete system architecture that cannot cope with increasing business changes. As decision-makers lack the skill to evaluate system architecture, which results in the delay of necessary system renewal, engineers are occupied with trouble-shooting and band-aid solutions, with no chance to learn new technologies or to design systems with ‘clever’ architecture. The third spiral relationship between user corporations and SI services vendors is hence beautifully completed.

Breaking out of the spiral: look at the forest, don’t just move around the trees

To summarize, Japanese corporations lack concrete IT strategies and the ability to envision the appropriate enterprise architecture that aligns with their business needs. As a result, SI services vendors adopt a ‘body shop’ strategy that gives no incentive for engineers to polish their skills. Japanese software vendors are not encouraged to provide solutions with the latest architecture that meets the needs of the global market. Being locked into such a vicious cycle, even the most talented engineers have very little opportunity to develop their skills to a world-class level.

The key which will unlock these spirals is to recognize that a shortage of talent is a structural problem, and this is rooted in how multiple industries, corporations and research institutes interact. In other words, the solution requires the cooperation of multiple corporations and institutions. Strengthening IT education alone can do very little. To break out of the spirals would require the following:

1. User corporations should take the lead in further integrating IT vendors of strong consulting capability into their operation. Enterprise IT architectural strategy should be taken into consideration at the early stage of a business strategy formulation. Corporations should adopt the use of objective and scientific evaluation methods to choose IT projects that can help to differentiate a business from the competition, instead of focusing on the cosmetic details of the screens and minor usability improvement. The ability of user corporations to generate business value by leveraging IT is hence the key to break out of the spirals.

2. Both user corporations and vendors should aggressively employ foreign IT talents that can hardly be developed under the current industrial environment. The influx of skilled workers will not only help resolve the shortage but can further stimulate the morale of the Japanese IT workforce.

3. Long-term-wise, improvement in how the academic and industries work together as well as closer cooperation with professional bodies such as the Project Management Institute is necessary. Organized effort is required to build a breeding ground for talents and to further pursue and extend the body of professional knowledge. IT management is a young field that requires synthesis of multiple areas of research and industrial knowledge. It has not yet settled on a solid foundation and its future will depend on how the industry can cultivate professionalism that is comparable to those of more mature professions such as a doctor, accountant and lawyer.JI

James Mok

James Mok, Msc, MBA, Beng, a Hong Kong-born Chinese, arrived in Japan as a researcher at the Institute of Industrial Science of Tokyo University in 1992, after completing graduate studies at Stanford, CA and Bristol, UK. He spent 5 years working as an engineer on the shop-floor of a Japanese manufacturer before switching to an enterprise IT career, to do better financially. He ended up as a rolling stone, switching identities as e-commerce consultant for a Japanese SI company, ERP consultant at a US consulting company, IT manager of a global manufacturer and then started the Japanese subsidiary of a US software vendor making its way into the ‘scared frontline’—the Japanese manufacturing shop floor.

Technical Director, Apriso Japan
james.mok@apriso.com

J@pan Inc Magazine, Nov/Dec 2007