English education is compulsory for all Japanese students from the first year of middle school through the end of high school. And yet, if you visit Japan, you'll find almost no one speaks English. Let's talk about why!
First--the education system. Most Americans are used to hearing about how the East Asian countries whoop our asses at math. It's true that students here are doing fairly sophisticated calculus that Americans might not encounter until college, if at all. (I dropped pre-calc halfway through my junior year of high school and never looked back.) The stereotypically Japanese method of rote memorization and practice, practice, practice is probably well-suited to math and science, and basically any subject where you can commit accepted facts, and how to apply them in accepted situations, to memory.
What I find troublesome is that this same method is used for language learning. I teach the first year reading classes, and their textbook consists of chapters that are basically stories or articles on a particular topic that also introduce certain grammar points. I teach them pronunciation of new vocab and phrases, reading comprehension, and some writing. There is very little speaking involved, and no creative work from them--they are not asked to apply the English they learn to making their own sentences or dialogues, for instance. Not to mention when an ALT isn't present, English classes are conducted almost entirely in Japanese.
The English exams are exactly what was covered in class--the stories are reprinted with phrases missing for the students to fill in, and basic questions about the content. So basically if they pay attention at all or do their homework, they should have no problems. In fact, the composition portion of their last test was an assignment I'd given them in class, corrected, and returned--so all they had to do to pass was memorize the corrected version of their homework and re-write it for the test. Now how on earth will they know what to do if they're approached by a foreigner who needs directions somewhere? Maybe one of their readings included a "giving directions" dialogue, but unless that dialogue is word-for-word what this random foreigner needs, they won't be able to help. Never mind actually going to a foreign country themselves and trying to navigate.
Second--the culture. It seems to me that it is unbearable for a Japanese person to make a mistake. They believe in practice makes perfect, never mind that what they are practicing might not be particularly useful or applicable to real life. Jonathan has told me about how the basketball club at his school will do the same drills ad infinitum until they are flawless, but are utterly unprepared for the chaos of an actual basketball game. So it doesn't matter how flawlessly you can do a particular maneuver on the basketball court, or how good your pronunciation of the word "ubiquitous" is, if you are unable to improvise in a random, real world situation.
This focus on very specific perfection means my students are mortified if I ask them a question they weren't expecting in class, because that means they haven't rehearsed the answer and so may make a mistake. It seems to me the bulk of language acquisition happens when you are verbally trying to express yourself, making mistakes, and learning from them. But if you are so humiliated at the idea of garbling grammar or mispronouncing a word, you're obviously too inhibited to speak naturally, right? Part of the aversion to learning this way may also be that most Japanese people are not used to interacting with foreigners with imperfect Japanese language skills. As a New Yorker I'm accustomed to communicating with people with heavy accents and broken English, both of us trying our best to understand. Whereas in Japan if you mispronounce a single syllable in a word you may get looks of total incomprehension. They're just not used to bridging that gap. The students here think it is the height of hilarity when I make a mistake; yesterday in my English club the kids were cracking up that I'd written けんど (kendo) instead of けんどう (kendou), the correct spelling. I was like, I didn't laugh at you when you spelled lacrosse "rakkros." (OK, I laughed in my head.)
Anyway, it is may be seen as ethnocentric for a foreigner who isn't a real teacher to criticize another country's methods... but the proof is in the pudding. Odds are that if you meet a Japanese person who can speak English, it means they spent some time living in a foreign country, NOT that they gained their fluency within the Japanese system.