Embedded Web Testing
The web testing puzzle is a jigsaw with 3 pieces, and for a long time the 3rd piece has been hiding down the side of the sofa, but I think Simon Stewart just found it. He's called it WebDriver - it needs dusting off, but it's looking like it's going to be the right fit.
The problem with testing websites is that it has always been messy. It's bound to be hard coordinating 3 tasks (browser, server, set of tests) across at least 2 processes on different hardware setups, and maybe even different platform architectures.
The set-of-tests piece of the puzzle is well understood; JUnit, TestNG, blah blah.
The well behaved server piece is solved for Java by Jetty - you can fire up an embedded server as easy as new Server(); (example).
Now we can make browsers be as well behaved - we can remote control web browsers from within a Java program. WebDriver currently supports Internet Explorer, Firefox and HtmlUnit for totally embedded work.
WebDriver driver = new FirefoxDriver();
driver.get("http://localhost:8080/dwr/simpletext/index.html");
driver.selectElement("id=demoName").setValue("World");
driver.selectElement("id=demoSend").click();
// Give the browser a chance to do the onclick
Thread.sleep(500);
WebElement reply = driver.selectElement("id=demoReply");
assertEquals("Hello, World", reply.getText());
Scott McMaster has some more examples.
Just to be clear HtmlUnit has existed for a while, and it's great for testing the sunny day case where browsers work according to a spec. It's complete enough to be able to run DWR for example, but it won't help you test the weird corner cases. WebDriver helps by reusing the weirdness directly from the browsers.
Update: Patrick Lightbody points out that Selenium Remote Control does something similar. The core Selenium controller class actually looks to be more capable than the WebDriver version, I try it out to see if it embeds as well.
DWR news round-up
A couple of weeks ago I recorded a Webinar on DWR (see previous post), which TIBCO are kindly hosting as a flash movie. It contains an overview of Reverse Ajax, integration with the Open Ajax Hub and TIBCO GI, and a new call center demo that will be part of DWR 2.1.
Sys-Con have published an article: "Simple Streaming AJAX Comet App with Open Ajax Hub, TIBCO GI, and DWR", that I wrote with Kevin Hakman. The main article link is here or you could try the print version.
Philip McCarthy has written an excellent article on "Writing scalable Comet applications with Jetty and DWR":
The Comet pattern allows you to push data to clients, and Jetty 6's Continuations API lets your Comet application scale to a large number of clients. You can conveniently take advantage of both Comet and Continuations with the Reverse Ajax technology in Direct Web Remoting 2.
If you are interested in how Jetty saves threads under heavy Comet load, it's well worth a read.
Conferences
I don't normally blog every time I go to a conference because I'm sure I would sometimes forget, and it's probably not that interesting (or is it? argue in the comments), however there are a few that are coming up:
NFJS Exchange: London / August 29-31
No-Fluff-Just-Stuffs are cool, and I particularly like that they are coming to the UK after being a US only event for years. What's good is the size - only 250 attendees, so it's way more intimate than JavaOne or something similar.
Plus there's always good swag at NFJS conferences, the swag for TRWE (below) is well known, but I'm told there's a free Wii to all attendees just for quoting a code: NFJS-DWR667. Register here.
I'm talking on DWR and on Ajax Security. I may also be sharing some stage time with Mark Goodwin.
The Rich Web Experience: San Jose / September 6-8
TRWE has an awesome list of speakers - I am looking forward to hearing Aza Raskin again. He was great at the Ajax Experience last year. I've not seen any conference that lets attendees choose from a Wii or a 30Gb iPod Video before, plus if you book using the nfjs2007speaker200 promo code you can get $200 off the entrance price. Register here.
I'm talking on DWR and on Ajax Security.
Future of Web Apps: London / October 3-5
FoWA is very different for me because it's not a full time developers conference. They cater for developers, but also to a range of other skills, which means there are a whole bunch of interesting speakers that I've not met before. I'm talking on Comet and DWR. Also interesting is their web-app for registered attendees, so you can see who else is going (obviously, you need to register to attend to see), and free beer road trip.
Grails Exchange: London / October 17-19
The Grails Exchange is a little strange because although there's there's a lot of Grails actions, there's also quite a lot of non-Grails-centric stuff going on too. I'm talking on DWR and on Ajax Security. If you're quick there's still early-bird registration.
The Ajax Experience: Boston / October 24-26
TAE / Boston is a way off; I've only just come home from TAE in San Francisco, so I'm way ahead of myself. We've not even arranged subjects yet.
Fixing browser security: SameRefererOnly
Web security is horribly broken, and lot has been said about CSRF, XSS, DNS-Pinning, etc, but not enough about what we can do to fix the mess.
I think we could adapt an idea like HttpOnly to tackle CSRF - I'd like to see a "SameRefererOnly" marker for cookies.
SameRefererOnly is an indication that a cookie should only be sent to a Site when the referring domain matches the destination domain.
A number of people have commented that you could use server based referer checking to fix CSRF, however that doesn't work for 2 reasons, firstly sometimes referers are not sent, and secondly using old versions of Flash, you can forge referer headers anyway.
However if we move the checking into the browser, then we should be able to instruct browsers to be more careful what they do without our cookies.
The official spec for Set-Cookie is RFC 2109/2965 which says this:
Set-Cookie: <name>=<value>[; <name>=<value>][; expires=<date>] \
[; domain=<domain_name>][; path=<some_path>][; secure]
Microsoft's HttpOnly spec adds this to the end:
[; HttpOnly]
I propose that we further add this:
[; SameRefererOnly]
Any cookie marked SameRefererOnly MUST only be sent to servers when the domain of the cookie matches the referring domain. The cookie MUST NOT be sent when there is no referring domain (this might help to prevent CSRF / Phishing mash-ups).
It's worth noting that this protection breaks in the presence of XSS flaws in a site, because XSSed commands come from the compromised domain, however that does not make this protection useless.
Plug-ins like Java and Flash that have their own HTTP pipelines generally don't send cookies so these systems probably do not need to be patched.
This proposal has 3 places where it can come in handy:
- As an additional defence-in-depth layer of protection for a site that already has other CSRF protection.
- On an Intranet site where access through browsers that implement SameRefererOnly can be mandated
- In a future world where enough browsers implement this, that other browsers can be excluded.
Note on spelling: The pedant might note that this should be spelt SameReferrerOnly - i.e. with a double 'r', and the pedant would normally be correct however this misspelling is so common that the HTTP spec makes it, and hence we all follow unquestioningly.
I'm sure there must be some holes in this suggestion, but I make it anyway so someone can move the discussion on.
JVM Usage Stats
Often when you open an XML config file in a DTD/Schema aware editor, and you're connected to the internet, the editor will reach out over the net to fetch the DTD/XSD, which gives webmasters some interesting usage stats.
I've commented before on how you can use this technique to look at the big growth in DWR usage, but I noticed that you could also use it to suss out some JVM usage stats because when Eclipse etc come read the DTD they use the Java version number as their User Agent.
Anyway, the numbers:
Caveat - there are about a million reasons why this is un-scientific. It may be interesting to some though.
Aside: I wonder if Java should adopt the 'standard' formatting for the User Agent string that is employed by all major browsers?
Free Webinar
The word Webinar may be one of the top-10 most hated words on the net (according to a bizarre and probably meaningless poll) but I'm not sure there is a better phrase that isn't either long or ambiguous*.
Anyway - I'm doing one.
The 'live screencast' is about DWR and TIBCO GI. It's a demo of creating a website with a rich set of widgets (courtesy of GI) and rich interactivity (courtesy of DWR).
I've spoken at a few conferences using the battleships demo, and felt that the game was interesting but not very real-world. This 'online conference' shows something that is very definitely real-world: a call center - but also one where there is some scope for innovation.
The 'broadcast talk' planned for next Wednesday, July 18, at:
- 10:00 am Pacific
- 1:00 pm Eastern
- 6:00 pm UK
- 7:00 pm Central European
It's organized by TIBCO and we'll be broadcasting using Webex, so you'll need to register.
* Wikipedia redirects Webinar to "Web Conference", but it's not a whole conference, just a seminar, and "Web Conference" makes me think of a physical conference on the web like TAE or TRWE rather than a conference that is broadcast using the web. Ho hum.
American Airlines using DWR
So you begin here:
And you then get the standard airline search screen. But with a new "Price and Schedule" feature. Pick that ...
The results page uses DWR to fetch extra details on a flight.
Update: I've just spoken to Steven Leija who leads the Ajax effort (he blogged about it here), apparently, the development was very smooth, and the feature is getting great feedback. Thanks for the heads-up Steven.
I've said this before, but it bears repeating - DWR is fantastic as a way to add Ajax features to an existing website without re-writing the whole thing from scratch. Walmart is another great example of this.
It's one more step on the road to total world domination that I talked about with Marla Parker over at java.net.
Google's next Open Source project
A prediction: In a few months time Google will release a project called 'GDT'. GDT is somewhat similar to GWT except that rather than compile Java to HTML+JavaScript, it will compile to Swing. So rather than being Google Web Toolkit, it will be Google Desktop Toolkit. You'll then be able to take the same source, and get out of it either a desktop app, or a web app.
While we're on the subject, the same idea in reverse: I wonder if anyone at Google is working on having JavaFX compiled using the GWT compiler?
Unofficial DWR BOF
TIBCO have been very kind to DWR in supporting me to continue development and in publicizing it too. They've arranged to have a slot for us on their booth at JavaOne:
Thursday, 12:00 to 1:30pm: If you are around at JavaOne it would be really good to see you. Simply turning up to say hi and thanks to the TIBCO people would be cool. DWR is far more alive now, thanks to TIBCO than it would be without them.
Thursday, 1:30pm onwards: Is there any interest in an unofficial BOF to discuss feature priority for the next release?. There are plenty of sofas in the pavilion. I suggest we meet at the TIBCO booth, and then find a place to talk about where we are going and what we should attempt for version 2.1. If you are interested, please mail [joe at getahead dot org], so I know who might be coming.
Don't forget the official DWR technical session, TS-6410: Hands-on DWR, or see my earlier blog post.
One of the things we have planned for version 2.1, is deeper integration with TIBCO General Interface, so we can build on the current integration. I hope, for example, that we can configure DWR to be a conduit between JMS and the Open Ajax Hub.
DWR Version 2.0 Released
After much waiting, we can at last, say - DWR Version 2.0 has been released.
There are a lot of new features in version 2. Probably too many because it took a while, and it's going to take a few weeks for the documentation to catch-up. Version 1.0 and 1.1 had very good documentation, and we're determined not to let it slip.
Reverse Ajax particularly is doing a lot of very hard things. It has to cope with buggy browsers, buggy web-servers, buggy proxies, multi-threading, multiple languages, and the fact that the web was never designed to do this in the first place. There may be some bugs in the edge-cases, but we'll be fixing them soon. If you do find anything wrong, please report it.
