Saturday, May 28, 2011

My situation recently: hot trains, workaholic and PHP

It's been a couple of months since I've posted anything here. The situation in Tokyo is getting better. Everything is almost back to normal. The only main difference is that we have to be careful on how to use electric appliances. With Fukushima's nuclear central down, we are facing power shortage. In trains, the air conditioning is not as low as usual. Train companies were told to keep it at least 2 degrees higher. It's already getting pretty now. I don't want to think about riding on trains this summer...

The reason why I've not been posting here is that I've been very busy since April. I'm working late every day, so I can't find much time to read programming books now. I've received Seven languages in Seven weeks, but I haven't opened it yet. I also wanted to buy The Joy of Clojure and a book about Cloud Computing, but I finally gave up.

I'm working on a web project, in Java(JSP,servlets,EJB), for a securities corporation (brokerage house?). The servlets/EJB part is hidden in a proprietary framework, so web pages can be created without any servlets/EJB knowledge. Most of the java classes and the JSP files are automatically generated. We are using Excel to generate XML files, and an Ant script to generate Java classes from the XML files. It's easy to generate, but the learning curve is steep. To make one screen, with a search button, a save button and a back button, it takes about 15 Excel sheets to write. The worst part is that it's not 100% automatically generated. Some of the XML files have to be manually edited, and some of the generated Java classes also have to be manually edited. After editing a Java class, we have to merge any new changes to avoid previous changes to be overriden. This is a bit annoying.

My company told us that there were more and more PHP projects available, so I decided to take a step away from Java and learn PHP. I intend to take the Zend PHP certification in September. I've bought Beginning PHP and MySQL to begin with. I played a bit with PHP ten years ago, but never had any opportunities to use at work. It was quite popular in France at the moment, but not in Japan. After all these years using Java, it's exciting to start something different.

Monday, March 21, 2011

Mardi 22, de retour a Tokyo

Les trois jours de repos de le Kansai nous ont fait du bien. Hier soir, les trains pour Tokyo etaient pleins. Depuis que de la fumee a ete apercu sortant du reacteur 3, on n'a pas enormement de nouvelles. On n'en connait toujours pas la raison. Les employes de Tepco ont repris le travail a 8h ce matin. On attend toujours avec impatience de savoir si le courant va pouvoir etre retabli dans le reacteur 2. L'armee a envoye deux chars pour faire le menage autour des reacteurs. Ils sont arrives sur le site hier soir. Reste a savoir quand et comment ils vont s'y mettre.

J'etais etonne en lisant cet article du magazine Le Point : "Dans la nuit de mercredi a jeudi, le gouvernement francais avait ordonne aux 120 sauveteurs de quitter les lieux de leur mission aux alentours de l'aeroport de Sendai et de rejoindre la base americaine d'Isawa, a 300 kilometres au nord." Dans un sens, ca se comprend, les risques sont bien reels la-bas. La securite des sauveteurs passent avant tout. Ce que je ne comprends pas, c'est la maniere dont tout cela a ete gere. Les sauveteurs francais sont arrives lundi 14 au soir a Tokyo. La situation douteuse de la centrale nucleaire de Fukushima etait deja connue. Deux jours plus tard, on leur dit de partir.

Par contre, bonne nouvelle, la France va envoyer une aide supplementaire contenant du materiel en relation avec la radioactivite, et de l'aide humanitaire contenant des couvertures, des bouteilles d'eau, des masques respiratoires, de la nourriture et des medicaments (plus de details ici).

Saturday, March 19, 2011

Samedi 19, Nara

Me voila a Nara, a l'ouest du Japon, chez ma belle famille. L'ambiance dehors et dans les trains est assez differente de celle a Tokyo. On ne sent pas la meme inquietude ni la meme fatigue chez les gens ici.

Vendredi, la climatisation a ete coupe au bureau, pour prevenir une eventuelle coupure de courant. La journee finie, je suis alle rejoindre ma femme a la gare de Tokyo pour prendre le train direction Kyoto. Nous avons la chance de pouvoir souffler un peu pendant trois jours. Les dernieres nouvelles ont enfin apportees une lueur d'espoir. Plusieurs groupes de pompiers venant de Tokyo, d'Oosaka et de Yokohama viennent preter main forte aux equipes d'intervention sur le site de Fukushima. La derniere methode de refroidissement du reacteur 3 semble porter ses fruits. On attend maintenant avec impatience de savoir si le courant va pouvoir etre remis en route demain. On croise les doigts.

Cet apres-midi, on est alle faire un tour au Mr Bricolage du coin. Plus de lampe torche !! Comme je l'ai ecrit dans un autre post, on a beau etre a plus de 500 kms de la centrale nucleaire, les gens achetent tout ce qu'ils peuvent ici aussi.

Je viens de regarder le 20h de France2. Mince, le Japon est passe en deuxieme position derriere la Libye. C'est que les nouvelles sont plutot rassurantes maintenant. L'envoye special avait l'air tout decu. Ils ne se sont pas fatiguer non plus d'interviewer les japonais pour savoir ce qu'ils pensent de la situation. Ca doit vendre moins quand les japonais sont (un peu) moins effrayes.


Mise a jour: je l'ai retrouve, le pitre de France2. Alain de Chalvron. Voir ce blog pour le decouvrir. Au lieu de l'envoyer au Japon, envoyez le au Salon de l'Agriculture la prochaine fois, a moins qu'il ne s'inquiete des gaz a effets de serre degages par les vaches...

Thursday, March 17, 2011

Jeudi 17, un retour en sueur

Comme je l'ai ecrit dans un autre post, les compagnies de train ont ete forcees de reduire leur consommation d'electricite. Le lundi 14, le resultat en fut chaotique. Les gares etaient noires de monde. Les interminables files d'attente s'allongeaient bien loin au dehors des gares. J'ai eu la chance de ne pas avoir eu besoin d'aller au bureau ce jour la, et franchement, quel soulagement !

Vu le chaos du lundi, le gouvernement a demande a la compagnie d'electricite de Tepco de revoir son analyse, et de faire en sorte que le nombre de trains puissent etre augmentes. Depuis, la situation s'est ameliore. Certaines lignes sont toujours arretees certe, mais les grands axes vers Tokyo tournent en moyenne a 80%.

Cependant, Jeudi 17, nouveau probleme. La temperature exterieure a baisse. Il fait assez froid, on imagine donc que beaucoup de monde va mettre en route les radiateurs electriques. En debut d'apres-midi, Tepco a annonce qu'il se pourrait qu'une coupure generale ait lieu en debut de soiree. Au bureau, vers 16h45, on nous a annonce qu'a cause des risques de coupure, il vallait mieux eteindre tous les ordinateurs, et rentrer chez soi ! De plus, les compagnies de train ont annonce qu'elles allaient une fois de plus reduire le traffic. Rentrer a 17h ? Du jamais vu.

On est tous parti a 17h. Bien-sur, la plupart des compagnies ont fait la meme annonce, donc enormement de monde s'est rue vers les gares au meme moment. C'est la premiere fois que je vois autant de japonais marcher plus vite que moi ! J'ai pu prendre les trains sans trop attendre, mais ce fut ma seule consolation. Je ne me rappelle pas avoir ete dans un train aussi plein. A Shinagawa, deuxieme gare apres Tokyo, la plupart des gens sur le quai ont du abandonner la partie et attendre le prochain train. Une femme derriere moi ne pouvait visiblement pas resister a la pression et s'est retrouvee ecrase contre moi. Mon voisin avait les mains sur les vitres pour eviter de tomber sur la personne assise devant lui. A chaque depart et a chaque arrivee, le mouvement du train nous emportait vers l'avant ou vers l'arriere. A chaque virage on se retrouvait soit ecrase, soit emporte. Le calvaire a du durer une bonne demie-heure. J'en suis sorti le dos trempe et mal aux bras.

Ce vendredi matin, retour a la normale. Mon premier train etait plein, mais rien a voir avec ce qu'on a vecu la veille. Les autres trains n'etaient pas si bondes. A se demander si certains ne se sont pas deja refugies ailleurs. Au bureau, ils nous ont coupe la climatisation. Il fait un peu chaud, mais pas de quoi se plaindre.

Lundi prochain, c'est jour ferie. Certains de mes collegues ont pris leur vendredi pour avoir quatre jours. Moi j'en profite pour aller a l'ouest chez mes beaux-parents avec ma femme, et voir de loin si la centrale nucleaire va tenir ou pas. Au moins, on va eviter les repliques quotidiennes et les coupures de courant. Je ne sais pas si c'est juste une impression, mais il me semble avoir vu pas mal de monde avec des gros sacs. A croire qu'ils vont eux-aussi passer le weekend loin des risques.


PS1 : Dans les infos en France, comme a l'ambassade de France, ils nous conseillent d'aller au sud. Nous on dit "aller a l'ouest". Sur l'ile principale du Japon (Honshuu), il y a plusieurs grosses "regions", dont le Kansai (ouest, ou se trouve Oosaka) et le Kantou (est, ou se trouve Tokyo). On dit donc "aller a l'ouest", et non pas "aller au sud".


PS2 : Je tiens a repeter qu'il ne faut pas croire toutes les conneries qu'ils racontent en France pour semer la panique. Ils cherchent le sensationnel. Faut que ca pete. On sent tout de suite la difference de mentalite entre ici et la-bas. Hier au 13h de France2, on voyais bien la difference entre le specialiste gardant son calme et expliquant que tout n'etait pas joue, et le golio de presentateur qui cherchait absolument a savoir quand ca va sauter.


PS3 : J'en ai des choses a dire ! Je vois tellement de conneries dans les infos francaises, ca me degoute... Ce midi, le pire article que j'ai pu lire est celui-ci : "Nous avons aucune information depuis l'explosion ! On veut savoir la verite !". Il est complementement panique le pauvre. Qu'est-ce qu'il fait encore ici ? Il n'a pas trouve le chemin de l'aeroport ? Rectification. Des infos, on en a en continue sur toutes les grandes chaines. On a un bandeau autour de l'ecran montrant toutes les dernieres infos. Que ce soit a propos de la centrale, des trains, des disparus... On peut voir en direct les conference de presse des politiques et des employes de Tepco. Dans les journaux, on a des comptes rendus detailles des evenements. Est-ce que tout est dit ? Personne ne sait. Mais de dire qu'on a aucune info ???

Faire les courses un lundi matin

Le dimanche 13 au soir, la compagnie d'electricite Tepco nous a annonce qu'il allait falloir reduire l'utilisation d'electricite. Pour ce faire, ils ont contacte les compagnies de train, et ont decide que les trains allaient devoir etre fortement reduits. A temps normal, les trains pour aller au bureau sont pleins. On n'ose imaginer ce qu'il va se passer si on en reduit le nombre. Finalement, on apprend que certains lignes sont purement et simplement bloquees pour la journee, et que d'autres lignes allaient tourner a 30% ou 40%. De chez moi, trois des cinq lignes principales pour Tokyo ont ete bloquees, ce qui veut dire qu'il est quasiement hors de question de partir aux heures de pointes, a moins d'etre suicidaire. Heureusement, le client chez qui je travaille m'a contacte pour me dire que je pouvais rester chez moi pour la journee. J'ai contacte ma compagnie pour lui faire passer le message, et on m'a dit que 70% de nos employes ne pouvaient pas aller au bureau.

Faire les courses un lundi matin. Pas n'importe quel lundi matin. Le premier lundi apres le tremblement de terre. Le premier lundi apres qu'on nous ait annonce qu'il y aurait des coupures de courant programmees. On commence a se dire qu'il vaudrait mieux avoir des bougies, des allumettes ou des briquets, des torches electriques, etc... Tout le monde pense la meme chose bien-sur. Le resultat ? Plus rien dans les magasins.

Les magasins ouvrent a 10h, je suis donc parti un peu avant 10h. Premiere etape, Conan (une sorte de Mr Bricolage). A peine arrive, je vois des employes devant l'entree, ce qui est mauvais signe. Ils nous annoncent que le magasin est ferme, et qu'il ne savent pas si il va ouvrir.

Deuxieme etape, le supermarche. Il y en a deux pres de chez moi. Je me suis d'abord rendu au premier, mais il etait deja trop tard. Des queues immenses devant les caisses. J'ai fait le tour des rayons et me suis apercu que tous les produits de premiere necessite avaient disparus, deja entasses dans les paniers des femmes au foyer qui ont du poirotter une heure ou deux devant le magasin avant l'ouverture. Je voulais juste acheter du tofu, mais pas chance, il n'y en avait plus. Au moins je n'y suis pas alle pour rien car j'ai pu retirer de l'argent.

Changement de direction, j'ai file au deuxieme supermarche. Je ne m'attendais pas a un miracle, mais ce fut pire que prevu. Il venait juste de fermer ! Les derniers clients etaient toujours a l'interieur a paqueter leurs precieuses denrees. Ce lundi, une coupure de courant etait prevu a partir de 12h20 dans mon quartier. Ils ont donc decide de fermer assez tot pour pouvoir faire le rangement necessaire. J'ai entendu une femme dire qu'elle allait verifier au magasin ou j'etais alle plus tot. Je lui ai dis que ce n'etait pas la peine de se fatiguer.

J'avais du temps, je suis donc alle au conbini (petit magasin ouvert 24h/24), juste pour voir. Comme prevu, plus de cup ramen. Par contre, il restait quelques onigiri.

Bref, je suis revenu les mains vides. Heureusement, nous avions quelques bougies et de quoi manger. Puis on nous a prete une lampe torche et d'autres bougies, donc pas de quoi s'affoler. J'ai pu acheter un briquet le lendemain pres du bureau. Depuis lundi (on est vendredi), nous n'avons pas eu une seule coupure de courant, bien qu'il en soit prevu une tous les jours, a heures variables.

Les japonais n'ont pas l'air de paniquer ici. Et pourtant. Ils stockent tout ce qu'il peuvent, meme les choses inutiles. Il devient difficile de faire des courses a cause de ce phenomene de stockage. Le gouvernement a meme demande hier aux habitants d'arreter de stocker des denrees inutilement. Plus de bouffe, plus d'electricite, plus de gazoil. Les japonais ne paniquent pas, et pourtant...Aux infos hier, on a pu voir que les magasins situes a l'ouest subissaient egalement la meme parano. Certains font cela pour envoyer de la marchandise a leurs proches habitant a l'est ou au nord. Mais certains font cela pour eux. Une femme disait qu'elle achetaient juste pour soulager son angoisse. Elle habite pourtant loin de la zone a risque... Elle aurait bien besoin de faire un stock de bon sens.

Wednesday, March 16, 2011

Mon vecu du seisme de Fukushima, depuis Tokyo

Je traduis en francais le contenu de mon precedent post. (Je n'ai pas d'accent sur mon clavier)

  • Je travaille a Tokyo dans un immeuble de 24 etages. Le 11 Mars, 14h46, j'etais au bureau au moment ou le seisme s'est produit. Nous en avions deja eu un le Mardi ou Mercredi precedent, j'ai donc pense qu'il s'agissait encore d'une autre secousse "habituelle". Puis ca s'est mis a trembler de plus en plus fort, de droite a gauche. C'est de loin le plus fort seisme que j'ai vecu en 10 ans de vie au Japon. Certains collegues ont commence a se refugier sous leur bureau. C'est la premiere fois que je vois ca, c'est dire la force de la secousse. Moi, je lacais mes chaussures, tout en gardant un oeil sur la porte de sortie, pret a bondir.
    La secousse devint plus faible (je ne peux pas dire au bout de combien de temps. Ca m'a paru assez long, entre 15 et 30 seconds peut-etre). L'immeuble tenait toujours debout, personne ne semblait blesse ici. Nous savions que, cette fois, ce n'etait pas une autre secousse "habituelle". Nous nous demandions ou le seisme s'etait produit. Tout le monde verifiait les dernieres infos sur le net. Premiere question qui m'est venue: rester ou sortir ? Nous attendions les instructions du centre de prevention des desastres. L'annonce arriva quelques secondes apres, nous informant de l'origine du seisme, de sa force, et de ce que nous devions faire. La consigne fut de garder son calme car nous etions en securite a l'interieur. Puis nous eumes plusieures repliques plus ou moins fortes. Une premiere, une deuxieme, une troisieme... L'immeuble tanguait a chaque replique. On se sentait comme sur un bateau, ce qui eut tendance a jouer un peu avec nos nerfs.
  • Puis une autre secousse assez forte, bien que moins forte que la premiere. J'ai dit a mes collegues que je sortais et qu'ils feraient mieux de venir. Nous sommes sortis, et attendions que les choses se calment. Il y avait beaucoup de monde dehors. Nous etions tous en train de verifier les infos sur nos telephones portables. Nous vimes ce qu'il s'etait passe au nord, et ce qu'il advint aux villes broyees par le tsunami. Il y eut des infos comme quoi le tsunami allait arriver sur Tokyo, bien que beaucoup plus faible.
  • Au bout d'une heure, nous sommes retournes au bureau. Bien sur, les ascenceurs etaient hors d'usage. Nous avons utilises les escaliers de secours pour remonter, et decouvrimes l'ampleur des degats dans notre immeuble : des fissures sur les murs et sur le sol. Cependant rien de serieux, bien qu'on en vienne a se demander si le batiment va toujours resister au cas ou un autre seisme puissant se declenchait.
  • De retour sur ma chaise. Il est presque 16h. Nous continuions de jeter un oeil aux infos, tout en travaillant. Tous les trains etaient arretes. Les lignes du metro egalement. Ca allait etre une longue journee avant de pouvoir rentrer. Certains collegues quitterent le bureau des que la sonerie de fin journee se fit entendre. Certains ont probablement du essayer de se trouver un chemin dans les gares bondees de monde. Certains ont decide de marcher jusqu'a chez eux, peu importe la distance. Trois heures ou plus de marche dans la nuit et le froid ne les ont pas arretes (une de mes collegues est arrive le lendemain matin a 4h30). En ce qui me concerne, j'habite beaucoup trop loin du bureau pour pouvoir marcher. J'ai decide de passer la nuit au bureau, a moins que les trains se remettent en marche. Ou a moins que les repliques sismiques ne soient trop fortes.
  • La nourriture ! Je n'y ai pas pense immediatement. C'est en voyant une photo d'un magasin ayant ses rayons vides que j'y ai pense. Une autre photo aussi, d'une femme portant quatre "cup ramen (nouilles)" dans ses bras. J'ai file en bas pour faire des achats. Il ne restait plus grand chose. Seulement des petites boites de "cup ramen". Les plus grandes etaient deja vendues. Plus de onigiri (riz), plus de obentou (boites repas), plus de pain. Heureusement, le peu qui restait me suffit largement.
  • Au bureau, il fait chaud. On a de l'eau chaude, les toilettes. Pas de quoi se plaindre compare a ceux qui vont devoir passer la nuit dehors ou dans les gares. Pendant la nuit, la East Japan Railway (compagnie de train de l'est du Japon) a annonce que ses trains bougeraient a partir de 7h du matin. J'ai decide de partir a 6h30 pour voir si pouvais prendre un train. J'ai deux choix pour pouvoir rentrer : l'un etant d'aller a la gare de Tokyo, et d'essayer d'attraper un des trois trains allant vers chez moi. L'autre choix etant de prendre la ligne Ginza du metro, d'aller jusqu'a Shibuya et de changer de train. Choix difficile. Le second est un plus long trajet. J'avais un mauvais presentiment a propos du premier choix, parce que la compagnie East Japan Railway a tendance a dire des bobards, ou plutot de relativiser un peu trop les problemes. Neanmoins, j'ai decide de suivre mon premier choix et d'aller d'abord a la gare de Tokyo.
  • Il m'a fallu trente minutes de marche jusqu'a la gare de Tokyo. Des personnes ayant passe la nuit la-bas etaient assises ou dormaient encore par terre. La plupart des magasins etaient fermes (j'en ai vu un seul d'ouvert ou il y avait deja la queue). Contre toute attente, il n'y avait pas beaucoup de monde a l'entree par laquelle je suis venu. Tres peu de monde. Un des passages principaux menant a plusieurs quais etait etonnement assez vide aussi. Par contre, les quais, eux, etaient bondes. Je ne pouvais pas distinguer ou la file d'attente se terminait. Deux des trois quais ou je pensais aller etaient noirs de monde. Entre temps, j'ai remarque qu'il y avait beaucoup plus de monde venant des entrees opposees a celle par laquelle j'etais venu. Tout le monde s'affairait a essayer d'acheter un billet de train, ou a se frayer un chemin vers les quais. Le dernier quai ou je projetais d'aller n'etait pas si sature, alors je decidais d'y rester.
  • L'attente fut longue. Sur le quai, un petit vent nous frigorifiait. Un employe de la East Japan Railway nous ressassait toujours la meme chanson : "Veuillez patientier. Le train est en cours de verification et devrait bientot arriver. Le terminus sera la gare X, etc...". Encore et toujours la meme annonce. Une des personnes pres de moi lui demanda d'arreter de dire des mensonges. Certains personnes vinrent a bout de patience et redescenderent les escaliers. J'ai remarque que d'autres employes en bas empechaient que plus de personnes ne viennent sur le quai. Le train arriva finalement apres une heure et demie d'attente. Une fois dans le train, la premiere annonce fut : "Nous avons des problemes avec un des wagons. Nous devons proceder a de nouvelles verifications. Il va falloir du temps avant que le train ne reparte". La goutte qui fait deborder le vase. Je suis sorti, ainsi que d'autres passagers. Changement de plan: direction le metro.
  • Une fois sorti de la gare, j'ai marche jusqu'a la station de metro de Nihonbashi. Le metro etait un peu en retard, mais arriva finalement apres 4 ou 5 minutes d'attente. Il etait bonde, mais j'ai vu pire. Une fois arrive a Shibuya, j'ai change de train. Il n'y avait que des trains s'arretant a toutes les gares. Le trajet allait etre long, mais qui s'en plandrait dans cette situation ? Apres un dernier changement de train, j'arrivais enfin a mon appartement.
  • Finallement, je suis arrive a 11h. 4h30 de trajet, pas trop mal compte tenu les conditions. Je regrette d'etre alle a la gare de Tokyo, mais il fallait bien confirmer. Depuis, nous avons des repliques tous les jours. La centrale nucleaire de Fukushima devient petit a petit hors de controle, bien qu'il n'y ait pas de raison de paniquer. A propos de panique, je vais ecrire un autre post a propos de ce que j'ai vecu le Lundi 14 quand j'ai voulu faire les courses.

Au bureau, les videos etant bloquees, je ne voyais que des images des evenements. Ce n'est qu'une fois a la maison que je me suis rendu compte de l'ampleur du chaos et de l'horreur. Une vague poussant les voitures, les camions, les trains, les bateaux, les maisons comme de vulgaires jouets en plastiques. Ce que j'ai vecu a Tokyo semble tellement insignifiant compare a ce que sont en train de vivre les habitants au nord. La plupart des habitants vivants dans les zones sinistrees ont tout perdu.
Beaucoup ont perdu leur vie. Beaucoup ont perdu leurs parents, leurs enfants, leurs amis, leur maison. Il est facile de s'imaginer comment ca doit etre dans ce genre de situation, mais je pense qu'il est difficile de comprendre vraiment a quel point la douleur et le desarroi doivent etre forts.

J'en profite pour ajouter un petit commentaire. J'ai regarde le 20h de France2 hier sur le net. Un envoye special a Tokyo etait en train de resumer la situation ici. Il m'a bien fait rire. Il parlait de la secousse qu'on a eu avant-hier. Le pauvre, il etait tout pale. Son hotel a tremble pendant 15 a 20 minutes. Il etait mal. Il disait que dans ses circonstances les japonais se refugient sous les tables, mais qu'il n'imaginait pas faire ca alors que tout tremblait pendant un siecle. Il est ou son hotel ? Sur un paquebot ? J'etais situe plus pres que lui de l'epicentre de son "interminable" seisme. Ca a bien secoue quelques secondes. Oui, on s'est demande si ce n'etait pas la grosse replique dont on nous parlait. Mais 15 minutes ? 20 minutes ? Il etait tellement panique qu'il en a perdu la notion du temps ? Ce n'est pas etonnant de voir les francais paniquer et rentrer en France quand on entend des anneries pareilles. En parlant de rapatries, je ne sais que penser de tous ceux qui ont prit la poudre d'escampette (ils ne sont pas tous dans ce cas heureusement. Je comprends ceux qui vivent au nord, ceux qui ont des enfants...). L'autre qui arrive a Roissy tout sourire, et qui dit qu'elle va peut-etre retourner au Japon dans 15 jours, si tout va bien. La situation ici ne prete pourtant pas vraiment a sourire. Meme pas la honte. D'apres Fillion, on est passe de 5000 francais sur Tokyo a 2000. Les 3000, ils sont passes ou ? Ils font quoi comme boulot au juste ? Bref, la version "apocalypse" des medias francais me laisse vraiment perplexe. D'entendre des anes raconter n'importe quoi pour effrayer le public (j'imagine qu'il faut du sensationnel). On ne sait vraiment pas en qui croire. Une chose est sure, c'est qui si ca se passait en France, ce serait un beau bordel.

Earthquake experience in Tokyo

I was at the office when the earthquake happened on March 11th. Here is my sequence of events, from Friday 11th to Saturday 12th.

  • I'm working in a 24-story building. The earthquake started at the beginning of the afternoon (officially at 14h46). We already had one on Tuesday or Wednesday, so I thought it was "just another one". Then it started shaking more and more. It's the biggest quake I've experienced in ten years here. Some Japanese colleagues started covering themselves under their desk. I was putting my shoes on (I was wearing slippers) and looking right at the door, ready to rush out. The shake started to slow down. The building was still in one piece, and so were we. We all knew it was a big one, and wondered where it came from. Everybody was looking at the latest news on Internet. Leaving out or staying in ? We were all staying in there, waiting for instructions from the disaster prevention center. Then came the announce, telling us how big the earthquake was, where it happened, and what we had to do. The instructions were that it was safe to stay in, and there was no reason to panic. Then came aftershocks. One, two, three... The building was dancing. It felt like being on a boat, and this was getting on everybody's nerve.
  • Another rather big shake (not as big as the first one though) came. I can't remember the time. I told my colleagues I was going out, and told them they should come too. So we went out, waiting for things to calm down. There were many people in the streets. We were all checking the news on our cell phones. We saw what was happening in the north, and what happened to the cities eaten by a tsunami. There were reports that the tsunami would come to Tokyo, but that it would be rather small. Nothing compared to what annihilated the north-east coast.
  • We came back into the building after about an hour. Of course, elevators were stopped. We used the emergency stairs and witnessed how much damages the earthquake had made to the building : cracks on the walls and on the floor. Nothing serious, but it still leaves you wondering what will happen if another big one comes.
  • Back at my desk. It's around four o'clock. We're still keeping an eye on the latest news while working. All trains around are stopped. So are the Tokyo Metro lines. It was going to be a long day before being able to go back home. As the "end of work" bell rung, some colleagues were already leaving. Some would try to find their way through the crowded train stations, despite the transport paralysis. Some decided to walk, no matter how far they were living. Even a three hour walk in the dark and in the cold would not stop them (One of my colleague decided to walk too and told me she arrived safely home the following morning, at 4:30). As far as I'm concerned, I'm leaving too far away from Tokyo. So I decided to stay all night at the office, unless trains would run again, and unless some big shakes were being felt again.
  • Food. I didn't think about food at once. But then I saw a picture of a convenient store, with empty shelves. And another picture of a woman holding four cup noodles. I rushed for the convenient store downstairs. It was almost too late. Only a few small cup noodles left. Bigger ones were already sold out. No more onigiri, no more obentou, no more bread. Fortunately, I had just enough for the night and the morning.
  • It was warm inside, I had hot water, toilets. Nothing to complain about. During the night, East Japan Railway announced that their trains would start moving around 7am. I decided to leave the office at 6:30am and see how it turns out. I had two major routes : one was to go to Tokyo Station and try to get on one of the three lines going to my place. Another one was to ride on the Ginza Line (Tokyo Metro) to Shibuya, change there and get on a train to my place. Difficult choice. The second one is a longer route. I had a bad feeling about the first one. East Japan Railway often says things different from what the reality is. Nevertheless, I decided to check at Tokyo Station first.
  • A 30 minute walk to the station. Some people who spent the night there are sitting or sleeping on the floor. Most shops are closed. The entrance from which I came from was not crowded at all. Very few people. I could not believe it. One of the main passage to the several platforms was not *that* crowded either. But the platforms were ! I couldn't tell where the waiting line was ending. Two of the three platforms I was planning to go were overcrowded. While I walked from one to another, I encountered more and more people aiming at the platforms, or lining up to buy a ticket. They were pouring in from entrances located at the opposite side of where I came from. The third platform was miraculously not very crowded, so I decided to wait there.
  • A long wait. It was cold on the platform, light wind blowing. An employee of East Japan Railway keeps on telling us the same message, on and on. "The train is being inspected, it will take some time until it arrives, the terminus will be station X..." Again and again. One man told him to stop telling lies. Some people around me gave up and got back downstairs. I noticed that the railway employees were preventing more people to come, to avoid that the platform gets overcrowded. Despite the bad feeling, I'm still waiting. The train finally came after an hour and a half. I manage to get on the train, and the first thing we're being told is : "One of wagon is having some troubles and has to be inspected again. It will take some time until it leaves the station". That the last drop poured in my patience pot. I went out. Some others went out too. Change of plan : I go for my second option.
  • Out of Tokyo Station, walking towards Nihonbashi Metro Station. The train was a bit late, but finally came. It was a bit crowded, but I've seen worse. I safely arrived at the Shibuya Station and changed train. The next train I was riding on only had trains which stop at every station. No express. Did I bother ? Of course not. It was going to be a long ride, but as long as I arrive at destination... One more train change, and I was home.
  • Finally, I arrived home at 11am. Not bad, but I regret going to Tokyo Station first. Now we're having aftershocks every day. The nuclear plant in Fukushima is slowly getting out of control, although there's still no reason to panic. Talking about panic, I'll write another blog entry about how it was to go shopping on Monday 14th.

I saw pictures of the tsunami on Internet, but I only discovered the scale of chaos and horror after reaching home. Cars, trucks, trains, boats, houses being pushed by the wave like plastic toys. My earthquake experience seems so insignificant compared to what people are facing over there. Most people living in the destroyed towns have lost everything. Lots have lost their life. Lots have lost their parents, their children, their friends, their home. People are starting to share what they saw. There was a man who was looking for his wife and daughters, who were heading to one of his daughter's graduation ceremony. He found their car among the rubble. Looking into the car, he could see some feet and arms inside, but he was not allowed to confirm who was inside... It's easy to imagine how it must to be in such a situation, but I think it's difficult to understand how painful it really is.

Wednesday, March 9, 2011

Oracle database certifications in Japan or how to make more money from certs

I don't know if this is well-known, but the path to become an OCA/OCP/OCM is a bit different in Japan. First, the naming is different. There are four exams : Bronze, Silver, Gold, Platinium. Yes, one more exam. Before 10g or 11g (I don't remember), certifications in Japan had no relation with OCA/OCP/OCM. Now, Silver=OCA, Gold=OCP and Platinium=OCM. What about Bronze then ? That's where Oracle found a way to get more money from Japanese. Japanese people love exams. There are all sorts of exams in Japan. An exam about a famous manga written by Yokoyama Mitsuteru called sangokushi (very good by the way if you're interested in China's romance of the three kingdoms), an exam about fishes, etc... Moreover, IT companies in Japan are taking certifications very seriously. Certifications are almost must-have career objectives. So many IT employees are aiming at Oracle certifications. Ok, back to the Oracle database certification. Let's compare the path to OCA in Japan and in the US.

Becoming an OCA in Japan


Remember ? OCA is equivalent to Silver, and Bronze is a pre-requisite.

Step1 : Passing Bronze

Pre-requisites for Bronze. You need at least one of the following:

  • 11g SQL Basics I[11g SQLI] : 22260 yens
  • Bronze SQL Basics I[Bronze SQLI] : 22260 yens
Then, you can take the Bronze exam, which costs 22260 yens.

Step 2 : Passing Silver

With Bronze in your pocket, you're ready to go for Silver. This will cost you another 22,260 yens.

Congratulations, you've spent at least 66780 yens, which in the current rate is about 805 US dollars.

Becoming an OCA in the US


You need at least one of the following:
  • Introduction to Oracle9i SQL : 125 US dollars
  • 1Z0-047 Oracle Database SQL Expert : 195 US dollars
  • Oracle Database 11g: SQL Fundamentals I : 125 US dollars
After one of the pre-requisites is cleared, you can take the Oracle Database 11g Administration I, which costs 195 US dollars.

So the cheapest path is 320 US dollars.

How do you call that ?

Different countries, different rates. Yes. Still, 805 dollars vs 320 dollars. How do you call that ? Fraud ? Racket ? Whatever you call it, I only see it as a way to grab more money. Fortunately, most companies in Japan will pay the exam fees if you pass. Let's call it extra motivation then. Thank you Oracle.

Monday, March 7, 2011

Oracle Java Architect certification updated

From August 1st, some pre-requisites will be added to the Java Developer and Java Architect certifications (Details here). Basically, you'll need to have attend some training courses. Expensive ones. Like 3000 dollars. Ouch. But that's not all. That will be 3000 bucks for the basic Java programming course. Wait, wait. I already have a few Java certifications in my pocket, do I need to do a basic course ? It looks like so.

The approved trainings for "Oracle Certified Master, Java EE 5 Enterprise Architect" are:

  • Java Programming Language, Java SE 6
  • Object-Oriented Analysis and Design Using UML
  • Developing Applications for the Java EE 6 Platform
  • Developing Architectures for Enterprise Java Applications

I clearly don't want to take "Java Programming Language, Java SE 6". I'd prefer to take "Developing Applications for the Java EE 6 Platform" instead. But that course says :
Required Prerequisites:

  • Experience with the Java programming language
  • Familiarity with object serialization
  • Familiarity with relational database theory and the basics of structured query language (SQL)
  • Familiarity with the use of an IDE
  • Java Programming Language, Java SE 6 (SL-275-SE6)

What ? I need to waste money on that "Java Programming Language, Java SE 6" course ?

"Developing Architectures for Enterprise Java Applications" is even worse. You need both "Object-Oriented Analysis and Design Using UML" and "Developing Applications for the Java EE 6 Platform" (which implies that you also need "Java Programming Language, Java SE 6"). I leave you calculate how much it costs to take that course.

(the content of this blog entry originally comes from my post at Coderanch. Check that thread for some more up to date news on this topic.)

Thursday, March 3, 2011

Using Clojure in Vim - Part 2 : accessing REPL via Nailgun

I explained how to install the Clojure plugin for Vim, Vimclojure, in a previous blog entry. Now that I'm starting to get familiar with Clojure, I thought it would be nice to be able to access REPL from Vim. To achieve this, I downloaded the Nailgun client and server. Download them from the following locations:


Starting the Nailgun server

As explained in the Vimclojure help documentation in Vim (:help vimclojure), the server has to be started with clojure and clojure-contrib jars in the classpath. For example : "java -cp clojure.jar;clojure-contrib.jar;server-2.2.0.jar vimclojure.nailgun.NGServer 127.0.0.1". Actually, in the Vimclojure plugin archive, there is a "bin" directory with two batch files: ng-server (for unix) and ng-server.bat (for Windows). These batch files allow to launch the server easily. Create a ".clojure" file in the directory where you launch the batch from, and put all the necessary jars and classpath directories in it. One per line.


After launching Nailgun, a message should appear, telling you that the server is running :
NGServer started on 127.0.0.1, port 2113

Installing the Nailgun client

Unpack the client archive in a directory of your choice. Edit your .vimrc (_vimrc) file, and add or modify the following lines :
let vimclojure#WantNailgun = 1 
let vimclojure#NailgunClient = "D:\\nailgun-0.7.1\\ng.exe"
Replace the directory with the directory where you unpacked the client.

Using REPL in Vim

Launch Vim and create a new Clojure file. For example, simple.clj. I assume that it is saved in one of the directories of the classpath used for the Nailgun server.
(ns simple)
(defn hello [] (println "Hello"))
To launch a new Vim REPL, type <LocalLeader>sr. <LocalLeader> is defined by the Vim maplocalleader variable, which is '\' if not defined. Another useful command is <LocalLeader>sR, which will start REPL in the namespace of the current buffer (it uses Clojure's "require"). In the previous example, this would start REPL in the "simple" namespace. REPL will be opened in a new window, where Clojure can be used as usual. Very convenient. Check the following image to see the previous tiny snippet in action.




There are other useful commands, which are all explained in ":help vimclojure". One of them is <LocalLeader>lw, which executes Clojure's "doc" to the word under the cursor. It's very helpful to check the documentation of a function. Also helpful is <LocalLeader>sw, which shows the source of the word under the cursor. The preview windows opened by these commands can be close via <LocalLeader>p. Check the Vimclojure's help for more.

Wednesday, March 2, 2011

Clojure In Practice : Yahoo! Weather

I have finished reading Clojure In Action. Now it's time for some hands-on exercises. There are some practical examples in the book, but they did not interest me. A while ago, when I was reading Maven By Example, I came across the Yahoo! Weather RSS Feed, and I thought that reading RSS feeds was an entertaining way to make samples. I'll try to make a simple example of how to read the Yahoo! Weather RSS Feed in Clojure. I'm still new to Clojure, so there may be better ways to do it. But at least, let's make something work.

I was thinking of what was needed in order to make use of the feed :
  1. A way to send a request to the Yahoo! Weather RSS feed.
  2. A way to interpret the feed (XML).
  3. A convenient way to hold the weather data.

Sending a request to the Yahoo! Weather RSS feed

The url to the Yahoo! Weather RSS feed is http://weather.yahooapis.com/forecastrss. There are two request parameters which can be used to customize the type of weather report : the WOEID, which is the weather location, and the units for temperature (Celsius or Fahrenheit). (Check all details here).



The clojure.xml namespace has a convenient function to parse xml data from a stream or a URI: the "parse" function. Let's try it by getting a weather report where temperatures are in Celcius :

(ns kuriqoo.weather
  (:require [clojure.xml :as xml]))

(def yahoourl "http://weather.yahooapis.com/forecastrss?u=c&w=")

(defn get-feed [woeid]
  (xml/parse (str yahoourl woeid)))
Five lines of code in order to grab the feed !! Programming is dead :)



The "parse" function returns a struct-map which has bunch of :tag, :attrs, and :content keys to represent the content of the xml. Cool, but not too convenient to use. The next step will be to make it easier to get values from tags and attributes.

Interpreting the feed

The clojure.zip namespace has some interesting functions to navigate through trees. As we know, xml represent data in a tree of tags, and clojure.zip will help go through it. When I first saw that namespace, clojure.zip, I thought it had something to do with zip files. No, it contains helper functions related to zipper data structures. In brief (from here), a zipper is a data structure representing a location in a hierarchical data structure, and the path it took to get there. It provides down/up/left/right navigation, and localized functional 'editing', insertion and removal of nodes. Let's change the "get-feed" function to transform our data into a zipper :
(ns kuriqoo.weather
  (:require [clojure.xml :as xml]
            [clojure.zip :as zip]))

(def yahoourl "http://weather.yahooapis.com/forecastrss?u=c&w=")

(defn get-feed [woeid]
  (zip/xml-zip xml/parse (str yahoourl woeid)))
I've only added a required namespace and a call to "xml-zip". Programming is dead, really.



The clojure.zip namespace has some basic functions to help us read the feed. We can go up, down, left, right inside the tree of tags. There's an more convenient way to navigate through it : the zip-filter namespace of the Clojure-Contrib library. The doc says that zip-filter is a system for filtering trees and nodes generated by zip.clj in general, and xml trees in particular. There are some usage examples in the source file (xml.clj), so it's worth looking at. Let's use it. From Yahoo!'s documentation, we can see that the "rss" tag is the root tag of the xml document. It contains a "channel" tag which contains a few sub-elements.

By using the zip-filter namespace, we can easily access tags and attributes. For example, to get the city name of the weather report, we would use :
(ns kuriqoo.weather
  (:require [clojure.xml :as xml] 
            [clojure.zip :as zip]
            [clojure.contrib.zip-filter.xml :as zf]))

(-> (zf/xml1-> zipper :channel :yweather:location) (zf/attr :city))
where "zipper" is the data returned by our "get-feed" function. As you can see, it first grabs the "channel" tag, then the "yweather:location" tag, then the "city" attribute. The "xml->" function may returns a collection. Instead, the "xml1->" function returns the first element of that collection. It's more convenient to get tags and attributes values.

Gathering the weather data

There are several ways to hold data in Clojure. The way weather data is expressed in xml seems to fit with Clojure struct-maps. I'll make four of them : one for the units (temperate, distance...), one for the wind information, one for the astronomy informations (sunset, sunrise), and one for the actual weather conditions.

(defstruct units :temp :distance :pressure :speed)
(defstruct wind :chill :speed)
(defstruct astronomy :sunrise :sunset)
(defstruct condition :text :temp :date)
Then comes the function which grabs the data we're interested in :
(defn get-weather [woeid]
  (let [zipper (get-feed woeid)
        z-location (zf/xml1-> zipper :channel :yweather:location)
        z-units (zf/xml1-> zipper :channel :yweather:units)
        z-wind (zf/xml1-> zipper :channel :yweather:wind)
        z-astronomy (zf/xml1-> zipper :channel :yweather:astronomy)
        z-condition (zf/xml1-> zipper :channel :item :yweather:condition)]
    { 
    :city (zf/attr z-location :city)
    :country (zf/attr z-location :country)
    :units (struct-map units
                       :temp (zf/attr z-units :temperature)
                       :distance (zf/attr z-units :distance)
                       :pressure (zf/attr z-units :pressure)
                       :speed (zf/attr z-units :speed))
    :wind (struct-map wind
                       :chill (zf/attr z-wind :chill)
                       :speed (zf/attr z-wind :speed))
    :astronomy (struct-map astronomy
                       :sunrise (zf/attr z-astronomy :sunrise)
                       :sunset (zf/attr z-astronomy :sunset))
    :condition (struct-map condition
                       :temp (zf/attr z-condition :temp)
                       :text (zf/attr z-condition :text)
                       :date (zf/attr z-condition :date))
     }))
That's a big function. There must be a way to refactor it, but I'll leave that as my next exercise. Finally, let's make a function to print out a basic weather report:
(defn print-weather [info]
  (let [condition (:condition info)
        units (:units info)]
  (println "Weather at" (:date condition) "," 
           (:city info) "," (:country info))
  (println (condition :text) "," 
           (condition :temp)
           (units :temp))))

Now, let's see all this in action. I'm using the WOEID for Tokyo/Japan, which is 1118370.
user=> (use 'kuriqoo.weather :reload)
user=> (def info (get-weather 1118370))
user=> (println info)
{:city Tokyo, :country Japan, :units {:temp C, :distance km, :pressure mb, :speed km/h},
:wind {:chill 8, :speed 16.09}, :astronomy {:sunrise 6:09 am, :sunset 5:36 pm},
:condition {:text Light Rain Shower, :temp 10, :date Wed, 02 Mar 2011 12:29 pm JST}} user=> (:country info) "Japan" user=> (-> info (:condition) (:text)) "Light Rain Shower" user=> (print-weather info) Weather at Wed, 02 Mar 2011 12:29 pm JST , Tokyo , Japan Light Rain Shower , 10 C nil

That's it. A simple weather report in Clojure. There are two things I'd like to do next. First, the feed contains a weather forecast for the following day, which I'd like to get. Then, I need to refactor the get-weather function and find a better way to get the data.

Book review: Clojure In Action

There are already a few books on Clojure. I decided to go for one of the latest, Clojure In Action (the book was still in early access, finished, but not published yet).

The first seven chapters teach the basics. They are easy to follow. I felt that the chapter on concurrency was not clear enough. A lot of text about common concurrency issues, but no practical examples of the different Clojure features. Maybe in the sections that I skipped ? (more on that next). I just wanted to see one example using multiple threads and Refs, Agents and others.

Leaving the basics, chapter 8 shows how to do mocking and stubbing in unit testing. Then come four chapters using third party libraries and frameworks. I'm ashamed to say that I skipped most of their content (50 pages). HBase, Redis, RabbitMQ, Hadoop... That would have been fine with small libraries which do not require configuration or knowledge. This does not reduce the value of the book though. It's an "In Action" book after all. It is clearly stated on the cover that it will use these frameworks. If you are interested to learn more than just the language, this book contains a vast panorama of "in action" examples. I may come back to these chapters later.

Then, back to the core. Protocols, macros, etc... The modus operandi thing in the Protocols chapter confused me more than anything, although it's a good exercise at making macros.

I haven’t read other books on Clojure so I can’t compare this one to others. I'm pleased with this book, although I was interested only in the core functionality. I think it will give you enough grounding to get going with Clojure.

Monday, February 21, 2011

Clojure clj-record library and Postgresql

I came across a problem when using the clj-record library with Postgresql. When creating a new record in a table, a java.lang.IllegalArgumentException is thrown, despite the fact that the record was successfully created. The "create" function inserts and retrieves the inserted record, so it looked like the problem happened when fetching the newly inserted record. By looking at the source code for the "create" function, I found out that the "insert" function called by "create" calls "id-query-for", which is a multimethod. For postgresql, the function is :

(defmethod id-query-for "postgresql" [_ table-name]
  (str "SELECT currval(pg_get_serial_sequence('" table-name "','id'))"))

Calling Postgresql's pg_get_serial_sequence means that the id column must be called "id", and that it must be "serial". My table layout looked like that :
CREATE TABLE users
(
  id integer NOT NULL DEFAULT nextval('seq_users'::regclass),
  "login" character varying(10),
  first_name character varying(20),
  last_name character varying(20),
  "password" character varying(10),
  email_address character varying(50),
  CONSTRAINT pk_users PRIMARY KEY (id)
)
The column id was properly named "id", but its type was not serial, which was the problem. pg_get_serial_sequence returned null, so the record could not be fetched. I changed my table layout to look like this :

CREATE TABLE users
(
  id serial NOT NULL,
  "login" character varying(10),
  first_name character varying(20),
  last_name character varying(20),
  "password" character varying(10),
  email_address character varying(50),
  CONSTRAINT pk_users PRIMARY KEY (id)
)

This worked as expected. I don't quite like the fact that the column id is fixed to "id", nor the logic to fetch the inserted record...

Monday, February 14, 2011

Using Clojure in Vim

There are already a few plugins for well known editors to use Clojure. Anybody familiar with Vim will want to use it with Clojure. My environment is not complete yet, but here is what I am using so far :
I'm currently using Vim 7.3, TagList 4.5 and Exuberant Ctags 5.8.

VimClojure plugin

The installation is straight forward, and is fully explained in a readme file in the archive. The following settings has to added in your .vimrc (_vimrc) :
syntax on
filetype plugin indent on
I've also added the following two settings:
let vimclojure#HighlightBuiltins=1 
let vimclojure#ParenRainbow=1
The rainbow parenthesis might be distracting, but I quite like it this way.


I'm leaving the plugin in offline mode, so I didn't configure the Nailgun server. This allows to use nice features like dynamic documentation lookup and Repl running in a Vim buffer, but I'll configure that when I get more familiar with Clojure. For the moment, I use Repl in another window and load the file I'm editing using (load "filename").

(UPDATE: the explanations for configuring the Nailgun client and server are here)

Exuberant Ctags


Nothing more than downloading the latest archive, extracting it, and setting the directory where the ctags executable is into the path.

TagList plugin

One problem: the taglist plugin relies on the filetype detected by Vim and passes the filetype to the exuberant ctags utility to parse the tags. With the VimClojure plugin on, the filetype is set to "clojure", which is not recognized by the taglist plugin. In order to tell the plugin to use the Lisp syntax, the following line should be added to the .vimrc (_vimrc) file :

let tlist_clojure_settings = 'lisp;f:function'
A useful setting is "let Tlist_Exit_OnlyWindow=1", which will close the TagList window if the last file was closed.

In Vim, the ":Tlist" command shows the tag list in a new window.


That's it. With a couple of plugins, using Clojure in Vim becomes much more fun.To generate the help documentation of both plugins, just type ":helptags ~/vimfiles/doc"(or another path where the docs are) in Vim. Take some time to go through the help and check useful settings can be used.

Using Exuberant Ctags on the command line with Clojure files

The following steps are not necessary in order to use the TagList plugin. It should be done only if you want to generate the tags file for Clojure files from the command line.

I downloaded the latest version (5.8). Unfortunately, Clojure is not supported yet. But lisp is. Exuberant Ctags autodetects a file type by looking at its file extension. We can for it to use a different language by using the --language-force option (--language-force=lisp), but it's annoying to do it all the time. What can we do then ? It is possible to override the file mappings via the --langmap option. Let's see how it's done:

Check the supported languages

ctags --list-languages

Ant
Asm
Asp
Awk
Basic
BETA
C
C++
C#
Cobol
DosBatch
...
Clojure is not there.

Check the file mappings

ctags --list-maps
...
Lisp     *.cl *.clisp *.el *.l *.lisp *.lsp
...
The extension above are associated to Lisp.

Adding Clojure file extensions to the Lisp mapping

New extensions can be added via the --langmap option:
ctags --langmap=lisp:+.clj --list-maps
...
Lisp     *.cl *.clisp *.el *.l *.lisp *.lsp *.clj
...
To set it permanently, we have to set this option in the CTAGS environment variable (e.g. set CTAGS=--langmap=lisp:+.clj), or in one of the setting files used by Exuberant Tags (e.g. $HOME/.ctags)

After everything was set, looking at a tags file generated by "ctags *.clj" will confirm that the Clojure extension is properly recognized.

Monday, February 7, 2011

Oracle ROWNUM : What it is and what it's not

What is ROWNUM ?

In Oracle, ROWNUM is a pseudocolumn added by Oracle after the data has been selected. The ROWNUM will be incremented only if the selected row satisfies the condition clause. A pseudocolumn does not physically exist, but it behaves like any table column. The first row returned by a query has a ROWNUM of 1, the second row has a ROWNUM of 2, etc... The ROWNUM can be used to retrieve the top n records of a query. For example, to return a maximum of ten cats from an animal table, the query would look like:
SELECT * from animals WHERE type='cat' and ROWNUM < 11

Pitfall 1: Ordered results

The preceding query returned ten cats. But it doesn't mean that the same ten cats will be returned each time we execute that query, because the fetched records are not ordered. However, adding a simple ORDER BY does not work:
SELECT * from animals WHERE type='cat' and ROWNUM < 11 ORDER BY name
because the results will be first fetched and then ordered. To get the first ten results of an ordered query, we have to fetch records and order them first :
SELECT * from ( SELECT * from animals WHERE type='cat' ORDER BY name) WHERE ROWNUM < 11

Pitfall 2: Getting the nth record

We may think that using the ROWNUM allows to fetch the nth record of a query. For example, getting the second cat of the list :
SELECT * from animals WHERE type='cat' and ROWNUM = 2
Unfortunatly, this doesn't work. In the first paragraph of this article, you might have noticed that some words are in bold : the ROWNUM is set after records were fetched and will be incremented only if the selected row satisfies the condition clause. What's the problem with that ? Imagine that there are ten cats in the animals table. After fetching ten cats, oracle will try to assign a ROWNUM to the first record. The first record ROWNUM is 1. But the query fetches only the record whose ROWNUM is 2. The first record will be skipped, but the ROWNUM will still be 1. The ROWNUM will not be incremented. Thus, the second record ROWNUM is 1. And again, the "ROWNUM = 2" condition is not satisfied. Finally, the ROWNUM will never be incremented, and no records will be fetched. The equality condition will only work with the first record (ROWNUM = 1).

Pitfall 3: Getting all records after the nth record

Well, the problem is the same as with getting the nth record. The condition will never be met. Having a condition looking like "WHERE ROWNUM > 10" will never return anything. The first record whose ROWNUM is 1 will not satisfy the condition, so 1 will be used again for the next record, and so on.

Execution plans

It's interesting to look at the difference of execution plan between "ROWNUM < 11" and "ROWNUM = 2". The first condition uses a "COUNT STOPKEY" operation, which means that Oracle performs a count operation where the number of rows returned is limited by the ROWNUM expression in the WHERE clause. Even if there are one billion records in the animals table, only the first ten records will be read.
SELECT * from animals WHERE ROWNUM < 11
------------------------------------------------------------------------------
| Id  | Operation          | Name    | Rows  | Bytes | Cost (%CPU)| Time     |
------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |         |    10 |   850 |     2   (0)| 00:00:01 |
|*  1 |  COUNT STOPKEY     |         |       |       |            |          |
|   2 |   TABLE ACCESS FULL| ANIMALS |    10 |   850 |     2   (0)| 00:00:01 |
------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   2 - filter(ROWNUM>10)
On the other hand, the erronous "ROWNUM = 2" condition produces the following plan:
SELECT * from animals WHERE ROWNUM = 2
-------------------------------------------------------------------------------
| Id  | Operation           | Name    | Rows  | Bytes | Cost (%CPU)| Time     |
-------------------------------------------------------------------------------
|   0 | SELECT STATEMENT    |         |    80 |  6800 |     3   (0)| 00:00:01 |
|   1 |  COUNT              |         |       |       |            |          |
|*  2 |   FILTER            |         |       |       |            |          |
|   3 |    TABLE ACCESS FULL| ANIMALS |    80 |  6800 |     3   (0)| 00:00:01 |
-------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   2 - filter(ROWNUM=2)
We can see that Oracle goes through all records and tries to filter them using "ROWNUM = 2".

References

Friday, February 4, 2011

Oracle : Primary Keys and Non-Unique Indexes

I was asked by a colleague recently about "Unique and Nonunique Indexes" in Oracle Concepts guide :
Oracle recommends that unique indexes be created explicitly, using CREATE UNIQUE INDEX. Creating unique indexes through a primary key or unique constraint is not guaranteed to create a new index, and the index they create is not guaranteed to be a unique index.

Well, I was as puzzled as she was, by the following two facts:
  1. Creating unique indexes through a primary key or unique constraint is not guaranteed to create a new index.
  2. The index they create is not guaranteed to be a unique index.

Looking at the "PRIMARY KEY Integrity Constraints" of the same guide, it says :
Oracle Database enforces all PRIMARY KEY constraints using indexes. The primary key constraint created for a column is enforced by the implicit creation of:
  • A unique index on that column
  • A NOT NULL constraint for that column
Composite primary key constraints are limited to 32 columns, which is the same limitation imposed on composite indexes. The name of the index is the same as the name of the constraint. Also, you can specify the storage options for the index by including the ENABLE clause in the CREATE TABLE or ALTER TABLE statement used to create the constraint. If a usable index exists when a primary key constraint is created, then the primary key constraint uses that index rather than implicitly creating a new one.

The important part is "If a usable index exists when a primary key constraint is created, then the primary key constraint uses that index rather than implicitly creating a new one". I was under the false impression that set a PRIMARY KEY constraint on a table would create a unique index. Well, no, not necessarily. Oracle will reuse (thus not even create) a non-unique index for the constraint, when the columns on which the index is are the same as those for the primary key. This is also true for indexes having more columns than the primary key. As long as the columns used for the primary are the same as the first columns used for the index, the index will be reused. Unique or not.

Another thing to know is "Deferred Constraint Checking". From the Concepts guide :
You can defer checking constraints for validity until the end of the transaction.
  • A constraint is deferred if the system checks that it is satisfied only on commit. If a deferred constraint is violated, then commit causes the transaction to undo.
  • If a constraint is immediate (not deferred), then it is checked at the end of each statement. If it is violated, the statement is rolled back immediately.
What happens when a PRIMARY KEY is created as being DEFERRABLE ? Until the transaction is committed, it is allowed to have duplicate values. So a unique index cannot be used. Oracle has to use a non-unique index. Again, a primary key constraint is not necessarily ensured by a unique index.

The fun part is that :
  • If a unique index was used by a primary key constraint and that constraint is dropped, the index will also be dropped !
  • If a non-unique index was used by a primary key constraint and that constraint is dropped, the index will not be dropped.

References

Wednesday, January 26, 2011

Experimenting with JPA 2.0 and Spring 3.0.x. A libraries nightmare.

I was making JPA samples while I was reading Spring Persistence with Hibernate. I started to make everything from scratch. The book uses Maven to manage the sample project, but I'm not a Maven person, so I decided to leave Maven and do it the good old way : compile, see a ClassDefNotFoundException raised, find out which library I have to add to the classpath, download it, extract it, sometimes figure out which extracted libraries are not necessary, adding the necessary ones to my project... It sounds like Daft Punk's Technologic ! (compile it, break it, find it, get it, extract it, guess it, add it...) I finally made it, but I can't remember how long it took me to do all these steps. Too long for sure.

Which is why I started studying Maven ! Although I knew about Maven's dependencies management, I never actually used it. There are some good online books at Sonatype. I went straight to the Maven by Example book. This is a good book. I discovered that Maven is much more than a dependencies management tool !

From there, I made a new JPA/Spring project from scratch, using Eclipse's Maven plugin, and while looking at Spring Persistence with Hibernate's xml, I added the necessary dependencies. No effort. Maven did it all.

Here is my pom.xml, using Hibernate JPA, Spring 3.0.5(using transactions without aspectj) and H2Database. It looks big, but it's just a bunch of dependencies declarations, which are made even easier via Eclipse's plugin.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>

  <groupId>springwithmaven</groupId>
  <artifactId>springwithmaven</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <packaging>jar</packaging>

  <name>springwithmaven</name>
  <url>http://maven.apache.org</url>

  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  </properties>

    <repositories>
        <repository>
            <id>JBoss Repo</id>
            <url>http://repository.jboss.com/maven2</url>
            <name>JBoss Repo</name>
        </repository>
        <repository>
            <id>ibiblio mirror</id>
            <url>http://mirrors.ibiblio.org/pub/mirrors/maven2/</url>
        </repository>
        <repository>
            <id>jboss-public-repository-group</id>
            <name>JBoss Public Maven Repository Group</name>
            <url>https://repository.jboss.org/nexus/content/groups/public/</url>
            <layout>default</layout>
            <releases>
                <enabled>true</enabled>
                <updatePolicy>never</updatePolicy>
            </releases>
            <snapshots>
                <enabled>true</enabled>
                <updatePolicy>never</updatePolicy>
            </snapshots>
        </repository>
    </repositories>

  <dependencies>
 <dependency>
     <groupId>org.springframework</groupId>
     <artifactId>spring-core</artifactId>
     <version>3.0.5.RELEASE</version>
 </dependency>  
 <dependency>
     <groupId>org.springframework</groupId>
     <artifactId>spring-context</artifactId>
     <version>3.0.5.RELEASE</version>
 </dependency>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>3.8.1</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.hibernate</groupId>
      <artifactId>hibernate-entitymanager</artifactId>
      <version>3.4.0.GA</version>
    </dependency>
    
    <dependency>
     <groupId>org.hibernate.javax.persistence</groupId>
     <artifactId>hibernate-jpa-2.0-api</artifactId>
     <version>1.0.0.Final</version>
    </dependency>
    <dependency>
     <groupId>com.h2database</groupId>
     <artifactId>h2</artifactId>
     <version>1.3.149</version>
    </dependency>
    <dependency>
     <groupId>org.springframework</groupId>
     <artifactId>spring-tx</artifactId>
     <version>3.0.5.RELEASE</version>
     <type>jar</type>
     <scope>compile</scope>
    </dependency>
    <dependency>
     <groupId>org.springframework</groupId>
     <artifactId>spring-orm</artifactId>
     <version>3.0.5.RELEASE</version>
     <type>jar</type>
     <scope>compile</scope>
    </dependency>
    <dependency>
     <groupId>commons-dbcp</groupId>
     <artifactId>commons-dbcp</artifactId>
     <version>1.3</version>
     <type>jar</type>
     <scope>compile</scope>
    </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
            <version>1.6.0</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>jcl-over-slf4j</artifactId>
            <version>1.6.0</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-log4j12</artifactId>
            <version>1.6.0</version>
        </dependency>
        <dependency>
            <groupId>net.sf.ehcache</groupId>
            <artifactId>ehcache</artifactId>
            <version>1.6.2</version>
        </dependency>    
  </dependencies>
</project>

Tuesday, January 25, 2011

Book review: Pro Git

Pro Git

I've been using both CVS and SVN at work, and never had the chance to try Git. I decided to learn about Git, so I looked for a book. I found Pro Git's homepage, with the book fully readable online, in several languages. We don't often have the chance to read free quality books online, but this one is 100% free. After spending some time on the train enjoying the first three chapters of the book, I decided to buy it. It's well worth it. The author did a great job presenting Git's functionalities, and the major differences with other version control systems.

The main chapters on Git basics and branching are very easy to follow. The author explains all the major commands, showing the command line and the execution result. You can read the book without even trying anything (although I would recommend to download Git and try as much as possible). The chapter on distributed workflows is particularly interesting. It shows you how Git is used in a project, depending on the size (and other factors) of the project. Finally, the author presents the major Git tools, explains how to customize Git, how to use it with Subversion (would you do that?), and how Git works internally, which is very instructive.

The book is thin, and easy to read, so anybody can finish it in a short time. If you want to get started with Git, you can pick up this book without hesitation.

I have one minor complaint : the book size is a bit different from other Apress books I have (may depend on the print ?). It breaks the balance of my beloved book shelf :)

Friday, January 14, 2011

Confirming Quartz Trigger firing times (especially cron expressions)

The Quartz Scheduler is very useful when it comes to scheduling jobs. It offers a wide range of trigger options, including cron-like expressions. Cron-like expressions are a bit obscure to read, unless you use them often. These expressions are explained in the CronExpression and CronTrigger classes API, and in the online tutorial. I sometimes need to make sure that the triggers will get fired at the proper time. The TriggerUtils class allows to output all firing dates of a trigger between two dates : TriggerUtils#computeFireTimesBetween(Trigger, Calendar, Date, Date). The Calendar parameter here is not a java.util.Calendar, but a org.quartz.Calendar, which can be annoying when using both java.util.Calendar and org.quartz.Calendar in the same class.

Making a sample program to output all firing times between two dates is rather simple :

import java.text.ParseException;
import java.util.List;

import org.quartz.CronTrigger;
import org.quartz.Trigger;
import org.quartz.TriggerUtils;
import org.quartz.impl.calendar.AnnualCalendar;


public class ConfirmTriggerFireTimes {

 public static void main(String[] args) throws ParseException {
  new ConfirmTriggerFireTimes().showTriggerFireTimes();
 }
 
 public void showTriggerFireTimes() throws ParseException {
  CronTrigger trigger = new CronTrigger();
  // Fire at 10:15am every Monday, Tuesday, Wednesday, and Thursday 
  trigger.setCronExpression("0 15 10 ? * MON-THU");
  java.util.Calendar startDate = java.util.Calendar.getInstance();
  startDate.set(2011,java.util.Calendar.JANUARY, 1);
  java.util.Calendar endDate = java.util.Calendar.getInstance();
  endDate.set(2011,java.util.Calendar.JANUARY, 31);
  outputFireTimeList(trigger, startDate, endDate);
 }
 
 @SuppressWarnings("rawtypes")
 private void outputFireTimeList(Trigger trigger, java.util.Calendar from, java.util.Calendar to) {
  List fireTimeList = TriggerUtils.computeFireTimesBetween(trigger, null, from.getTime(), to.getTime());
  for ( int i = 0; i < fireTimeList.size(); i++ ) {
   System.out.println(fireTimeList.get(i));
  }
 }
}
In this example, I want to confirm the firing dates of a trigger set to fire at 10:15am every Monday, Tuesday, Wednesday, and Thursday in January. The resulting output is :
Mon Jan 03 10:15:00 JST 2011
Tue Jan 04 10:15:00 JST 2011
Wed Jan 05 10:15:00 JST 2011
Thu Jan 06 10:15:00 JST 2011
Mon Jan 10 10:15:00 JST 2011
Tue Jan 11 10:15:00 JST 2011
Wed Jan 12 10:15:00 JST 2011
Thu Jan 13 10:15:00 JST 2011
Mon Jan 17 10:15:00 JST 2011
Tue Jan 18 10:15:00 JST 2011
Wed Jan 19 10:15:00 JST 2011
Thu Jan 20 10:15:00 JST 2011
Mon Jan 24 10:15:00 JST 2011
Tue Jan 25 10:15:00 JST 2011
Wed Jan 26 10:15:00 JST 2011
Thu Jan 27 10:15:00 JST 2011
Mon Jan 31 10:15:00 JST 2011
This looks good. We can see that any jobs associated to this trigger will be executed at 10:15am every Monday, Tuesday, Wednesday, and Thursday.

Sunday, January 9, 2011

Book review : The Android Developer's Cookbook : Building Applications with the Android SDK

The Android Developer's Cookbook is a recipe-styled book, where each recipe shows how to use a particular feature of the Android SDK. Each recipe is more or less independent of the others. It's not a classical beginners book, but I think it can still be used to start learning about Android development. It starts with an overview of the Android platform, then presents various recipes in a logical order. First, the most basic recipes : activities, intents, threads, services, alerts, widgets and other ui, events like key presses and Touch events. Then recipes explaining how to use specific functionalities : multimedia, hardware (sensors), networking, data storage, location services like Google Maps, and many more advances recipes. Finally, recipes on debugging.

The authors are using Eclipse and its Android plugin to create sample applications. The book is very easy to follow. There are a lot of code snippets, and some pictures to illustrate their execution. Anybody with some basic UI understanding (e.g. Swing experience) should easily read through the content. It's the kind of book you'd keep on your desk for further reference. It's not a complete reference book though. Explanations and samples are short, so you may still have to look for more detailed information in the online documentation. It's a nice cookbook. Not complete, but well worth reading.

Tuesday, January 4, 2011

Book review : Spring Persistence with Hibernate(Apress)

Spring Persistence with Hibernate (Beginning)

If you are looking for a book to learn about Spring and Hibernate, pass your way. If you are looking for a reference, pass your way. So who is this book for ? I think it is aimed at people who want to try a simple application using Spring3 and Hibernate 3.x (JPA2). It is fast paced, straight to the point. If you know what you are doing, it's a fun book. You'll start by setting your development environment (authors use Maven), configure Spring and Hibernate, make some domain classes, make some DAOs... Very fun. But don't expect to find answers if you're stuck somewhere.

There are some interesting explanations about persistence optimization like caching and lazy-loading, as well as a chapter about integration of frameworks like Dozer and Lucene. It also mentions REST and Spring MVC, and concludes with Grails and Spring Roo. These last two might be out of topic, but they have their own merit. I think they are worth reading.

I didn't notice many typos. Source snippets are neither too short nor too big. They illustrate well the explanation they are attached to. I already know about Spring3 and JPA2, but I never used Hibernate as my persistence provider. This book provided me a chance to try it. I felt it was not like any other technical books. Very enjoyable.