The easy part of networking as a foreigner in Japan is getting together with people of similar backgrounds and cultures, and mixing for social reasons as much as business. Many of us do this without thinking, and without realizing that we're missing the best part of being in Japan - and that is the chance to mix with our Japanese colleagues. After all, associations and clubs are THE glue that holds Japanese business together and which allows CEOs and others to find new opportunities in this face-to-face culture.
Now I admit that trying to socialize outside your native tongue, and not knowing some of the rituals and procedures of Japanese business groups can be a bit daunting at first, even if you speak Japanese reasonably well. But my experience has been that as a foreigner in a Japanese business group, you may well be the first non-Japanese member they've had in quite some time, and thus the other members will treat you with tolerance and interest. And the bonus is that once you get past being an object of curiosity, you really can get some good business leads.
There are zillions of Japanese associations and clubs - pretty much one for anything that covers a business sector, not to mention old boy's groups, study clubs, etc. It's easy to get drawn into too many of these groups, so I suggest that you focus in on the following: a) the leading organization representing your sector of business, b) alumni associations of graduates from your college, c) Venture company groups, since these organizations are usually populated by the young and adventurous, and d) Leadership groups which might yield some useful high-end relationships.
As for any networking, the best place to start finding these organizations is on the Web, and perhaps the most prestigious group is the Keizai Doyukai (Japan Association of Corporate Executives). This august 1,400-member organization functions as a think tank for the captains of 900 companies and they prognosticate to the press on a monthly basis. They have a web site at http://www.doyukai.or.jp/. To be honest, I find them a bit too "aerial" for my taste, but some of my friends with loftier ambitions like mixing with them. But I guess if you'd like to polish your keigo while looking to bump into someone famous, then try to attend an event that combines with him or her with a foreign chamber or business group.
Another prestigious organization, and possibly a bit more useful is the Tokyo Chamber of Commerce and Industry (and its equivalent in other cities). Although there are about 85,000 members in TCCI, surprisingly few foreign firms belong, thus they always seem happy to invite foreigners to participate in their events. You can find them at http://www.tokyo-cci.or.jp. Their networking events are split up into both sectors and regional chapters, making them focused and drawing a decent spectrum of people in your industry.
If you're into Venture Capital, or simply like doing business with freethinkers in that sector, there are a few organizations around. A couple that I can recommend are the Tsunami Network Partners gatherings, which you can find through their http://www.tsunami2000.co.jp website, and the Young Entrepreneur's Organization (YEO) at http://www.access-t.co.jp/yeo/. Yoshito Hori, who today runs the Globis MBA college and the Globis Capital Partners VC fund among other things, established this second organization. There are over 180 CEOs of venture companies in the YEO, and although you need to be a CEO and founder to join, you no longer need to be young (they dropped the age limit) and it's not hard to get invited as a guest.
Then of course, there is the government, which is in any case one of the main drivers of Japanese industry through its assiduous use of regulation and deregulation. A good site to visit to get a feel for the scope of industry and semi-governmental organizations available to network with is at http://www.meti.go.jp/english/network/index.html. You can also check out the Small and Medium Enterprise Agency for METI, http://www.chusho.meti.go.jp/, which holds events of all kinds for Japan's 5 million+ smaller firms.
I'm of course only touching the surface here. But the point is that to get the most out of your networking efforts, expand yourself beyond your normal comfort zone and try to find some other groups to connect to. Japanese business works on the principal of familiarity, so set yourself a target group to work on, and start building some lifelong relationships. Over time, you'll find them paying off both business wise and socially.