How Joomla 1.5.6 came about
Coordinator Blog
Written by Anthony Ferrara   

As most of you know, a critical security vulnerability affecting all Joomla versions below (and including) 1.5.5 was discovered on Tuesday, August 12th 2008.  What most of you don't know, is what went on behind the scenes that day.  A whole mass of people came together and immediately worked on all the tasks necessary to make 1.5.6 happen.   Experiencing this first hand was quite amazing...  Publishing a release is a process that normally has two weeks (and a team of people) devoted to it (for everything from selecting which remaining artifacts will be fixed, to translations, to clicking publish and everything in-between).  This all happened in a VERY short time.

Here's an abridged breakdown of how 1.5.6 came to be...

15:50 EST

Bug Squad member Marijke Stuivenberg points the squad to a reported vulnerability in Joomla 1.5.5.

15:55 EST

Bug Squad members Jennifer Mariott, Elin Waring, and Marijke (along with development coordinator Wilco Jansen, OSM Vice President Rob Schley and myself) verify that the vulnerability exists and the report is valid.

15:56 EST

All available development Work Group members, Bug Squad members and Core Team members are notified of the issue.

Bug Squad confirms that 1.5's SVN is stable and is ready for immediate release pending vulnerability fix.

Forum moderators are informed of and asked to remove references of this issue until release.

16:05 EST

Patch is generated and provided to Bug Squad for testing/confirmation of fix.

16:20 EST

Patch is confirmed to fix vulnerability.

Front page announcement is drafted.

16:30 EST

Patch is committed into SVN along with all preparations for release. 

Joomla 1.5 branch is frozen for release cycle.  Bug Squad begins testing sanity and operation of SVN.

16:46 EST

Security announcement (on developer.joomla.org) is drafted.

17:20 EST

Front page announcement provided to translators.

Joomlacode prepared for release.

17:30 EST

Bug Squad confirms sanity of SVN and that all release preparations are in place.

Package generation begins. 

17:50 EST

Full download packages generated.

18:05 EST

Packages provided to Bug Squad for validation and testing.

18:30 EST

Bug Squad confirms package sanity, final steps before release are completed.

18:40 EST

Front Page article and Developer security report published. 

Full download packages released.

19:30 EST

All patch downloads tested and published.  Release cycle completed.

Conclusion

Total time from report of vulnerability to initial release: 2 hours 50 minutes

Total time from report of vulnerability to completion of release cycle completion: 3 hours 40 minutes

Total number of people directly involved: between 20 and 30

< Prev   Next >
 

Show other articles of this author

331 Votes

68 Comments

Feed

  1. avatar fairooz makes this comment

    Thursday, 11 December 2008

    Thanks so much.

  2. avatar James makes this comment

    Monday, 01 December 2008

    Hey Great Work...

    Did you notify the 200,000 website owners of the security problem?

  3. avatar addicting games makes this comment

    Friday, 28 November 2008

    I like this site.

  4. avatar Barcelona hotels makes this comment

    Tuesday, 25 November 2008

    I got hacked with the 1.5 ver. I hope the version will be more secure.

  5. avatar Joomla fan makes this comment

    Monday, 24 November 2008

    I love your work! Keep on like this!
    saludos desde España

  6. avatar meteo makes this comment

    Friday, 14 November 2008

    Joomla... what a wonderfull and crazy tool ! Thanks so much.

  7. avatar Fotos de Sexo makes this comment

    Thursday, 13 November 2008

    Excellent work!
    Great versions.
    Thanks

  8. avatar pothios makes this comment

    Wednesday, 12 November 2008

    cool thanks

  9. avatar Nick Kotarski makes this comment

    Monday, 10 November 2008

    The speed of the fix is pretty impressive.

    Nick

  10. avatar Mature makes this comment

    Monday, 10 November 2008

    Great job and teamwork everyone. That was an amazing experience!
    Thanks

  11. avatar Det makes this comment

    Wednesday, 05 November 2008

    What an example for companies sticking to bureaucracy.

    (When will they publish the movie?)

  12. avatar NBA makes this comment

    Saturday, 01 November 2008

    very nice

  13. avatar mil4o makes this comment

    Thursday, 30 October 2008

    Yep my website was hacked to because of this. Good thing i had a backup. Thanks to joomla team for making our lifes so much easier. Nothing can replace joomla ! It's just the best !

  14. avatar Altamiraweb makes this comment

    Wednesday, 15 October 2008

    Well done folks! but I got already hacked with the previous ver.

  15. avatar Fabio makes this comment

    Saturday, 11 October 2008

    Excellent work !!!!!, GOD BLESS YOU ALL! .

    Thanks,
    Fabio

  16. avatar Muhammad Ali makes this comment

    Friday, 26 September 2008

    Check out Abhay hitting on Amy hahaha funny as &*!).

    By the way amazing work guys...As it happens I had a 1.5.5 build on me and was installing it onto my brothers website. Just for kicks I visited the Joomla website and found 1.5.6 and read about the security venerability in 1.5.5.

    Thats all fixed now.

    And Abhay I hope you find a good girl friend and wont have to hit on women over the joomla developers comment posts.

    OHH P.S. Which comment component is this?

  17. avatar Kalemanzi makes this comment

    Saturday, 20 September 2008

    Well done guys and gals. We're all proud of you and proud of Joomla! Commercial packages are rarely able to achieve such fast security patches as this.

  18. avatar jorgerosa makes this comment

    Wednesday, 17 September 2008

    Amazing team!!! I doubt that commercial software developers could work so hard and faster. well done guys!

  19. avatar abhay makes this comment

    Monday, 15 September 2008

    Well Awesome Job By the team Amy can you please share your Email
    Mine is arora.abhay@gmail.com

  20. avatar Amy Stephen makes this comment

    Saturday, 13 September 2008

    Red - please share your email address if you wish to share your comments with others.

    Bradly - have you lost a baby? It's important to keep perspective; Web sites are not anywhere as important as people. If your point is that we, as a community, must continue to work hard to strengthen security, then your point is absolutely correct.

  21. avatar Stefan makes this comment

    Friday, 12 September 2008

    PLEASE Next time DO NOT PROVIDE THE DETAILS OF THE FIX.

    Because all the junior hackers will test and try to hack our Websites, your detailed fixed is silly, because it described to everyone how to hack a Joomla 1.5.5, the biggest threat is your detailed descriptions!!!!

  22. avatar bradly makes this comment

    Thursday, 04 September 2008

    Wow, all this high-fiving, back slapping, self congratulation seems to belie the fact that if proper code review was done in the first place, this wouldnt have happened in the first place??

    Kinda like a drunk driver that goes out and sideswipes several cars, runs over a baby but luckily..

    15:50 EST calls lawyer
    15:55 EST Lawyer looks up "HOW TO PERFORM ROADSIDE CPR" via GOOGLE, relates info to client
    15:58 As our hapless driver tries in vain to locate a banana to stuff up the car's tailpipe, they both realize the attorney has looked up "ROADSIDE CAPER" instead of road side CPR

    16:10 performs CPR on several auto tailpipes (to warm up, before attempting CPR on child.

    16:20 after failing, advises mom to place child in trunk, for 30 days, then claim child was given to babysitter and/or abducted.

  23. avatar Scott makes this comment

    Thursday, 28 August 2008

    Hi Guys, thanks for update. Is there a mailing list we can join to be informed of updates, etc?

    Thanks!

  24. avatar Romkabouter makes this comment

    Thursday, 28 August 2008

    Well done folks! Excellent

  25. avatar James makes this comment

    Monday, 25 August 2008

    I am amazed at how quickly you pulled off the new release. The speed at which everyone jumped in just shows how dedicated everyone on the team is to Joomla. Many of us out here using Joomla for free are obviously taking too much for granted.

    So ladies and gentlemen thank you VERY MUCH for your hard work.

  26. avatar Mark makes this comment

    Monday, 25 August 2008

    Thank GOD you guys got this out - My site was just hit over this weekend, as I wasn't aware of any updates.

    I'm patching it right now - and glad to see you guys working hard to fix these things!

  27. avatar Surya makes this comment

    Sunday, 24 August 2008

    Hey

    You guys doing great jobs. Keep it up. You are like gladiators in this open source world. Keep rocking. Let them hack again, we will kick their ass even more harder than this.

    Regards,
    Surya.

  28. avatar Janne makes this comment

    Friday, 22 August 2008

    Excellent response. Big hand to the team!

    The idea of a security team is great and a "critical fixes" email-list would be a good addition to that. Only this kind of "emergency" information could be shared instantly to the administrators joined this list.

    Do you, I mean we :), have a hacking team of our own? I mean people with knowledge of hacking methods to stress the system constantly.

  29. avatar Greg Owens makes this comment

    Thursday, 21 August 2008

    Hey Folks, Just wanted to say that one of my websites was attacked just yesterday Aug 20. I just found out about the update. I'm using a Shared Hosting provider and have used the automated installer. I have asked them when they are going to upgrade from 1.5.4 to 1.5.6 and am awaiting a response.

    A Big thank you to the team. What list do I need to subscribe to to the the security updates in a timely manor?

  30. avatar Joel makes this comment

    Thursday, 21 August 2008

    I'm not a techie, just someone who runs 3 Joomla-based sites, so I can only say how much I appreciate the work of others.

    Is there a way to build a background "check at launch" into the Joomla admin backend to check if a critical -- i.e. security -- update is required? Have it enabled by default, with optional user disabling. This might make a lot more admins aware of such issues more quickly. (It could also contain an option to advise on any version update, whether security-related or not.)

    If Firefox can tell me when one of my plugins has an update available, this shouldn't be all that difficult to implement.

  31. avatar alexio makes this comment

    Thursday, 21 August 2008

    Ultra super KUDOS for the fast fix up job, gang!!! I for one, heard about the problem thanks to the folks at JXTended who sent out an email describing the problem to their list. Perhaps you ought to institute a "Security Alert" email Signup that sends out to all Joomla admins when you find a breach?

  32. avatar Mike makes this comment

    Wednesday, 20 August 2008

    THANK YOU GUYS! GOD BLESS YOU ALL!

  33. avatar Andrew Eddie makes this comment

    Wednesday, 20 August 2008

    Daniel, even though we caught this one well, there is room for improvement. A security strike team is being assembled to be even more effective in alerting the community. Hopefully this will also allow them to react more quickly. We don't get these hum-dingers too often fortunately, but when they do come, we want to be better prepared.

  34. avatar Daniel makes this comment

    Tuesday, 19 August 2008

    My site was also hacked twice in 2 weeks. First time was using 1.0.x and second time using the latest 1.5. Why is this and how is it so easy to get a jomla site hacked. I am very concerned now and I am needing answers so I can stop my community site being hacked again. I can not afford to keep doing this as it is affecting my customers to.
    Well done on fast response but surely more can be done. It is not a isolated problem. I will even offer mys ervices to trying to find a solution if I can be of any help.

  35. avatar Roman makes this comment

    Tuesday, 19 August 2008

    Congratulations to the team, but... I agree with Marco's comment: response of Joomla! users is slow - particularly compared to the response of hackers. My Joomla! 1.5.2 site was hit by massive defacing attack yesterday, obviously "inspired" by recently published detailed description of the reset token validation mechanism flaw.

  36. avatar Smiley makes this comment

    Tuesday, 19 August 2008

    A good problem-solver team!

  37. avatar linuxway makes this comment

    Monday, 18 August 2008

    Thank you so much for all your efforts. This is truly an amazing team.

  38. avatar cjw makes this comment

    Monday, 18 August 2008

    One of my sites got hacked and apparently got used to try and get into the bank of Schotland. That's al least what my provider told me.

  39. avatar Jorgen makes this comment

    Monday, 18 August 2008

    Team, thanks for the fast turnaound of this issue. Next Time in wish the have a pre-announcement of a hot security fix commin along, so I can free up some time upfront and be prepared for a night activity.

    Jorgen

  40. avatar Doug makes this comment

    Monday, 18 August 2008

    Got Hacked as I was updating my last site on the 16th, managed to change admin password then logged out admin, finished uploading patch changed admin password again. Took me all day to clear problems (front page - login page - all forms - and clear out a number of .jpg's scattered around in different directories including root, purpose unknown.) Thanks to all for the quick patch, need some way to get out knowledge of any security patch quickly! First I knew was e-mail from Barrie North at Compass Designs thanks also to him for e-mailing his users!

  41. avatar Lucas makes this comment

    Sunday, 17 August 2008

    Congratulations, thanks to the team!

    Great Job.

    Regards
    Lucas

  42. avatar Amy Stephen makes this comment

    Sunday, 17 August 2008

    Alex777 - Why don't you help with the Bug Squad? There is no "you" - just "us"

  43. avatar alex777 makes this comment

    Saturday, 16 August 2008

    hacking the site shouldnt have been that easy, teenage lamers.. google search index.php?option=com_content.. pick up a joomla site.. 30 sec. to taking over..
    thats unbelievable..
    guys, forget everything, please focus on security
    if you lose someone's trust, there's no turning back
    dont sacrifice such a great project to a simple mistakes

  44. avatar Marco makes this comment

    Saturday, 16 August 2008

    Thank you so much for the hard work and devotion to the project. That is much appreciated.

    The only comment I wish to make to this article is that, YOUR timeframe from bug-to-fix is perfect. But OUR timeframe (the users of Joomla!) is significant lower. Currently there is no good way to stay on top of these urgents updates, e.g. mailinglist/xml feed with -only- latest version info.

    Just my 2 cents.

  45. avatar Livingstone makes this comment

    Friday, 15 August 2008

    Great job guys? But what exactly went wrong? Some dude has just recently changed the names and offline data of my joomla sites! I have no idea how he got into the admin backend? Is there any where this issue is properly documented or any treatment for victims? Just asking cause ill like to know to what extent this useless individual got into my site!

    I am upgrading now though...

  46. avatar Tobias Bornakke makes this comment

    Friday, 15 August 2008

    Got a hacked site, but so glad to see a fix so fast so that it only is one and not 10 sites we are dealing with!

  47. avatar John makes this comment

    Friday, 15 August 2008

    Congratulations and a very sincere thanks to the entire team!

  48. avatar Tony makes this comment

    Friday, 15 August 2008

    My site also hacked - not much done, fortunately. VERY glad to see such a quick response and release of a patch, excellent work and congratulations to all concerned. No clients affected, luckily, but great to so quickly identify the problem and so easily fix it. Thanks again!

  49. avatar JoeJoomla makes this comment

    Thursday, 14 August 2008

    That really is very responsive! I wasn't so lucky to escape being hit but it's good to see that you have a great team. I'm just sorry I am not in a position to contribute more than I do at this time. Anyone should be proud to be part of Joomla!

  50. avatar BADEsemowo makes this comment

    Thursday, 14 August 2008

    Wow !!! to think I was away when all this was going on. I missed the announcement... i got hacked 2 days later... but i guess I am happy that the community was on top of it before it became a scandal. Thank you people.... Thank you.

  51. avatar severdia makes this comment

    Thursday, 14 August 2008

    It was in miraculously record time this issue got resolved. Just imagine how much quicker it would have been if we'd all been sitting around doing nothing at the time? :)

  52. avatar Jaider Bertoli makes this comment

    Thursday, 14 August 2008

    thanks for the effort

  53. avatar Elin Waring makes this comment

    Thursday, 14 August 2008

    @admin evaluating

    You should subscribe to the security announcements forum. If you do you will get an email any time there is an announcement. Also you can take the RSS feed from the security center here.

  54. avatar bilgi makes this comment

    Thursday, 14 August 2008

    wow, what a great. it seems the fastest security fix on the world.

    thanks for your efforts

  55. avatar pdisanto makes this comment

    Thursday, 14 August 2008

    i really appreciate your efforts, thank you..
    but, my two sites are hacked using this bug before 1.5.6 released (and many others as i know)
    i think this bug is listed on well-known hacker's sites before patching.
    security is everything, somethimes it costs too much. for example, i've lost my two joomla customers yesterday..

  56. avatar Flavio Copes makes this comment

    Thursday, 14 August 2008

    Reading this report was quite like being there, thanks for making the rest of us join this effort :)

    Great work!

  57. avatar Wilco Jansen makes this comment

    Thursday, 14 August 2008

    Next time I whish this report comes a bit earlier so it won't cost me (and others) a lot of sleep...anyway, this was a really amazing effort from people all over the world. Well done all!

  58. avatar Tobias makes this comment

    Thursday, 14 August 2008

    I love your work! Keep on like this!

  59. avatar Admin Evaluating makes this comment

    Thursday, 14 August 2008

    I am evaluating joolma right now and there are three things that really should be addressed fast:

    1. why there is no Multisite Feature? How can I have a codebase for all sites? This is also very important for security updates.

    2. I see no security mailing list or any kind of notification where I could subscribe for security updates ONLY - a joomla sec announce list or something would be really very very useful and nowadays every project should come wit that.

    3. I can not search the site (THIS site). This is totally annoying.

    Please change these things fast so we can go on with our review. Thanks!

    Gabor Goldbowm

  60. avatar Pris makes this comment

    Thursday, 14 August 2008

    Is that security vulnerability affects 1.0.15 version?

  61. avatar julien makes this comment

    Thursday, 14 August 2008

    This is insane !
    I've never heard such short respond time to a problem.

    Actually you guys must be even faster than the emergency team at NASA !

    Impressive! And thank you so much for your effort to make sure end user like me don't go through boxes of paracetamol because of a exploited site.

    cheers

  62. avatar Tshidiso makes this comment

    Thursday, 14 August 2008

    Excellent work great people, we could be anywhere without this family. Keep it up. U guys are the greatest.

  63. avatar inglorius makes this comment

    Thursday, 14 August 2008

    my site was hacked

  64. avatar Flavio Adalberto Kubota makes this comment

    Thursday, 14 August 2008

    Excellent work!! This motivate everyone who is involved with Joomla! to help to improve it. Great job!

  65. avatar Amy Stephen makes this comment

    Thursday, 14 August 2008

    This is an amazing group of people who care very much about making Joomla! better. There is no company in the world that can motivate employees to produce better quality results and respond more quickly during a problem then a community with pride in what they do. Excellent job!

  66. avatar Elin Waring makes this comment

    Thursday, 14 August 2008

    Great job and teamwork everyone. That was an amazing experience.

  67. avatar Brad Baker makes this comment

    Thursday, 14 August 2008

    It was quite amazing to be apart of this. I will say as well, due to this swift action, potentially 10's of 1000's (or more) of websites were patched before they had a chance of being exploited.

    Great work everyone!

  68. avatar Michelle Bisson makes this comment

    Thursday, 14 August 2008

    Excellent work Anthony and Bug squad!

Add Comment


    • >:o
    • :-[
    • :'(
    • :-(
    • :-D
    • :-*
    • :-)
    • :P
    • :\
    • 8-)
    • ;-)



    Click to get a new image.