In the Spotlight - Bill Scott

Recently, I have been giving a <a href="http://billwscott.com/share/presentations/2007/rwe/Antipatterns.pdf">new talk on Anti-Patterns</a>.<br /><br /><a href="http://billwscott.com/share/presentations/2007/rwe/Antipatterns.pdf" ><img src="http://billwscott.com/share/bio/antipatterns-cloud.png" alt="current set of anti patterns" width="400" /></a><br /><br />Sometimes it is most instructive to look at design patterns in reverse-- as a set of anti-patterns. In this talk, I am exploring the common mistakes that designers &amp; developers make when attempting to craft a rich web experience. There are a bunch of counter-examples from consumer facing web sites (both inside &amp; outside of Yahoo!) as well as from enterprise web applications.<br /><br />The anti-patterns explored are:<ul><li>meandering way</li><li>borg idiom</li><li>tiny targets</li><li>hover and cover</li><li>pogo stick navigation</li><li>novel notions</li><li>metaphor mismatch</li><li>double duty</li><li>linkitus</li><li>windows aplenty</li><li>animation gone wild</li><li>misguided misdirections</li><li>missed moments</li><li>one at a time</li><li>non-symmetrical actions</li></ul>Venues for the talk so far have been at:<br /><ul><li><a href="http://ajaxexperience.techtarget.com/west/index.html">Ajax Experience, San Francisco, CA</a></li><li>Yahoo! Design Expo, Sunnyvale, CA</li><li><a href="http://www.therichwebexperience.com/conference/san_jose/2007/09/index.html">Rich Web Experience, San Jose CA</a></li><li><a href="http://www2.sys-con.com/ajax2007west/scheduleNEW.cfm">Ajax World, Santa Clara, CA</a></li></ul>Enjoy.

<img src="http://billwscott.com/share/blog/proto_logo.png"/><br /><br />Wow. It has actually been 2 months since my last post. Nope. I have not fallen off the planet. This has been one of the craziest times I have experienced in a long time so while I have had a lot to chat about I have not had the time to blog coherently.<br /><br />One of the projects I have been working on is a way to make it easier for all of the non-techies (as well as techies) to be able to experiment with various interactions on a web page in lightweight manner.<br /><br />By lightweight, I mean to say a way to take an existing web page or a prototype of a web page and "sprinkle" rich behaviors to play with concepts. Ideally you should be able to express an interaction in a declarative manner without having to write JavaScript directly (but not kept from it if you need to). And even more ideally you should be able to build up the "legos" of interaction in simple GUI tool embedded in the browser so that you are not encumbered with even a simple syntax to type in.<br /><br />So back in July, I was slated to give a talk at the Ajax Experience. I promised a few months before that I would have 2 new talks. One on Anti-Patterns (yes I owe several blog posts on that one) and the other on prototyping.<br /><br />The problem was exactly one week before both talks I had literally had not time to put those talks together. So I dropped out for a few days to create a very early version of what is now protoscript. The good news was that it was actually fully functioning and I gave a peek of it at the conference. <br /><br />I got a lot of good feedback from the talk and went back and rewrote it a couple of times, changed up some of my approach, moved to a plug-in style architecture, etc.<br /><br />So, ok, what is protoscript?<br /><br /><img src="http://protoscript.com/images/promo.gif"/><br /><br />Protoscript is a simplified scripting language for creating Ajax style prototypes for the Web. With Protoscript it's easy to bring interface elements to life. Simply connect them to behaviors and events to create complex interactions.<br /><br />The key abstraction that came out of my thinking was to express everything as objects (think HTML elements), behaviors (think patterns) and events (both events and callbacks).<br /><br />In some ways it is similar to jQuery. And in fact I use jQuery for the selecting of elements. Protoscript, however, is a very stylized way to describe interactions. This makes it ideal as a building block.<br /><br />For example, since the language is so simple (It is actually in JSON format) it makes it easier to extend to multiple toolkits (currently I use YUI, but I have built a simple example with Mootools), add new behaviors and events and even more importantly be able to build GUI tools on top of it.<br /><br />As a first hack I created Protoscripter a bookmarklet that brings a GUI interface to any web page allowing you to play with protoscript (without any installation, other than the bookmarklet) and experiment with interactions.<br /><br />I have been careful about pushing protoscript out because of this layering approach. Ideally I would have the whole stack. I would have a complete GUI as a FF extension allowing designers, product marketing or web devs to quickly sprinkle interactions onto their prototypes and quickly iterate to the best design. Well that is not reality. I have invested about 2-3 weeks of time so far and feel I have gotten it to the right level to release to the public.<br /><br />There are 31 behaviors, 40+ live demos, a simple GUI tool (protoscripter). The scripting engine seems to be pretty solid (knock on wood). I have a full wiki with full documentation on all those behaviors. The code is over in Google Code in a repository ready for open source contribution.<br /><br />So to kick it off I am giving a talk this morning at the Rich Web Experience here in San Jose. You can get a PDF version of my talk here: <a href="http://billwscott.com/share/presentations/2007/rwe/Prototyping-final.pdf">Prototype Presentation</a>.<br /><br />And most importantly start checking out <a href="http://protoscript.com">prototoscript.com</a> (right now the best experience is in Firefox).

I am having the hardest time filling two positions on my team. And I am working on what I think is one of the <a href="http://looksgoodworkswell.blogspot.com/2007/03/yahoo-teachers-and-yahoo-gobbler.html">coolest products in Yahoo!</a> (<a href="http://teachers.yahoo.com">Yahoo! Teachers</a>). I don't feel like I am asking for too much.<br /><br />Here is what I am looking for.<br /><br />Two passionate, talented software engineers with experience in developing web 2.0 social applications. While they must be proficient in PHP/mysql, I am looking for engineers that understand good software architecture, scalability concerns, performance engineering, and skills at assessing and implementing integration points with other Yahoo! platforms and properties. Experience with developing web services and/or search technologies is a plus. No need for front-end web development experience as the two positions focus on backend services and technologies. But of course any front-end experience is a plus.<br /><br />To apply you should have at least 3-5+ years of experience. The key is web application development. Not interested in software engineers that have built microcontrollers or mobile apps or embedded devices or written some html. Seriously. This is an incredible opportunity, loads of fun, very rewarding... So how about it? <br /><br />Both jobs are similar. Here are the postings:<br /><br /><a href="http://pljb6.rmx.scd.yahoo.com/pljb/yahoo/yahoo_jobs/applicant/jobClick.jsp?count=1&id=10295">Tech Yahoo, Software Apps Dev Eng, Sr</a><br /><br /><a href="http://pljb6.rmx.scd.yahoo.com/pljb/yahoo/yahoo_jobs/applicant/jobClick.jsp?count=1&id=10293">Tech Yahoo, Software Apps Dev Eng</a><br /><br /><br />Apply at the site. Also if you want drop me a line at: yteachersjobs [at] yahoo [dot] com.

I decided to wrap the <a href="http://developer.yahoo.com/yui/logger/">YUI Logger</a>, add a JavaScript command line that outputs the results in a hyperlink format to facilitate simple drill down.<br /><br /><img width="450" style="border: 1px solid gray; padding: 2px;" src="http://billwscott.com/share/blog/YUIDebuggerSnap.png" alt="YUI Style Debugger"/><br /><br />Works on IE6, IE7, FF. Haven't tested the other browsers.<br /><br />Just a quick hack to make some stuff I was doing go simpler. Use it if it helps.<br /><br />You can get it at <a href="http://billwscott.com/debugger/">my debugger page</a>.

<span style="font-weight: bold;font-size:100%;" >Ok, How About a Revolution?</span>Aren't you tired of getting a chunk of search results from Google, Yahoo, A9, product listings, real estate pages, etc.? And then having to scroll to the bottom and hit the "next page?". In the words of Andy Rooney, "I know am!" What I normally do is search, scroll through results to the bottom of the page, scroll back up to double-check, then I have to scroll back down to navigate to the next page. Seems like a lot of bother to just move through my results.<br /><br />Have you ever thought that there could be a better way -- or at least an alternative to the standard google paging control?<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://static.flickr.com/30/58655423_50eaf89e86.jpg"><img style="margin: 0px auto 10px; display: block; text-align: center; width:400px; cursor: pointer" src="http://static.flickr.com/30/58655423_50eaf89e86.jpg" alt="" border="0" /></a><br /><span style="font-weight: bold;">First a Little History</span>At Sabre, we started experimenting with managing very large data sets with HTML tables. Our in-house table component had grown very rich in features (things like inline editing, sorting, selection, multi-sort, custom editors, row re-arrangement, grouping and other desktop-like features). But one thing we were not happy with was its handling of large sets of data.<br /><br />Our table supported two styles of navigation: scrolling and paging. With scrolling you could create a natural way to interact with the data-- but if you had a lot of records just the amount of HTML that was required to download to the page made it unusable once you got over 1000 records.<br /><br />We supported paging early on to address this issue. The user chooses the page by selecting a specific page number or using the next/previous paging controls.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://static.flickr.com/28/58655407_89e5465d10.jpg"><img style="margin: 0pt 10px 10px 0pt; width:400px; cursor: pointer" src="http://static.flickr.com/28/58655407_89e5465d10.jpg" alt="" border="0" /></a><br /><br />One nice feature I added was a pre-calculation of the first and last field values of each page for the sorted column. I then displayed this in a page navigation combo box.<br /><br />This allowed the user to navigate to a particular part of the data without having to guess which page. Additionally, I added tooltips that showed what each page contained. This worked fairly well.<br /><br />However, most clients really wanted us to make the table a scrolling table. This was not feasible due to large data sets.<br /><br /><span style="font-weight: bold;">The Dilema</span><br />The catalyst came when we were designing a new web application. Initially all of the design discussions pointed to tables with less than 500 records. Since the user needed to select multiple values within the table we designed the screens to use the scrolling variety of our table. However, soon after implementation started the design requirements changed (!) and now we had to deal with 10's of thousands of flights.<br />Now we could have moved them to our paging table. However, given the requirement for the user to move through the flights and make multiple selections before moving to the next step, we were concerned with the usability of this approach. First, would users really think their flight would still be selected after paging to the next set of results? Seeing the page refresh does not make one think everything has been kept. Second, the user really wanted scrolling tables.<br /><br /><span style="font-weight: bold;">Enter Ajax</span><br />So what we settled on was to take the existing paging table and turn all page and sort requests into Ajax-based requests. Our theory was that if the transition between pages was almost instanteous without a page refresh then the user would be able to 1) understand selection was persistent, 2) be able to move through the data quickly. It turned out that performance was excellent. And since it was faster there was less confusion about whether selection was maintained while paging.<br />Darren James (on my team) did an incredible job of squeezing more and more optimizations out of the ajax response/update. It was his work on the table, cinematic effects and drag & drop that brought about the Rico framework.<br /><br /><span style="font-weight: bold;">User Feedback</span><br />The users absolutely love it! They couldn't believe the responsiveness they were getting from a table in a web application. But they still had one request. They wanted to control the table by scrolling. And they wanted it to be fast even with large data sets.<br /><br /><span style="font-weight: bold;">A Scrolling Ajax Table</span><br />So after launching Rico (<a href="http://openrico.org/">http://openrico.org</a>) Darren, Richard Cowin and I felt that this would be the right thing to tackle. We all felt that it should start as a lightweight behavior. <a href="http://openrico.org/behaviorFeatures.page">Behaviors</a> in Rico are just JavaScript logic and/or layout that gets applied to raw HTML. In this case we are applying just the logic of hooking up an HTML table to any arbitrary set of data via Ajax. Other behaviors (sorting, filtering, reordering, etc.) are planned later.<br /><br /><span style="font-weight: bold;">Introducing LiveGrid Behavior</span><br />By applying the LiveGrid behavior, your HTML table gets connected to any size data set and it gets adorned with a scrollbar to control the navigation. The scroller becomes the replacement for the standard Previous/Next page controller bar. When dragging the scroll bar you are actually issuing ajax commands to pull the data into the table in real time! Of course we provide data caching and event buffering strategies to minimize the number of requests and smooth out the scrolling.<br /><br /><span style="font-weight: bold;">Death to Paging?</span><br />First, let me say that I really don't believe that paging is evil or that it is wrong in all cases. But I believe the reason that most sites have paging solutions is because 1) its easy to implement, 2) handles the problem of large sets of data and 3) it is convention.<br /><br />Rarely is paging decided because it is the better choice over something like scrolling. (I should hasten to add that I do discuss some of the cons of scrolling later in this article.)<br /><br /><span style="font-weight: bold;">Our Pitch</span><br />So, here is what we are advocating. We would like people to start thinking about replacing the paging data model with the more inuitive scrolling data model. It is not an accident that for years this has been a good model on the desktop. And there have been a number of tables built in the desktop or applet world that do exactly what LiveGrid does-- connect up to a virtual set of data and allow you to scroll through it. In fact, I wrote a table to do this very thing in 1998 for the Java Swing framework.<br /><br />So we are hoping the release of LiveGrid will create a dialogue of discussion around the best way for users to move through information on the web and in web applications. We need better ways of displaying search results, product lists, real estate listings, etc. on the web.<br /><br />To help you understand some of the possible impact we provide two demos for the scrolling LiveGrid.<br /><br /><span style="font-weight: bold;">Pitch #1: To the Corporate World: You Can Do It on the Web!</span><br />The first example demonstrates a table with around 1000 movies accessible. Only ten are shown and possibly 30 are cached on the browser at any given time. By scrolling the table, the database is queried to return more data just in time and on-the-fly.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://static.flickr.com/28/58655407_89e5465d10.jpg"><img style="margin: 0pt 10px 10px 0pt; width:400px; cursor: pointer" src="http://static.flickr.com/28/58655407_89e5465d10.jpg" alt="" border="0" /></a><br /><br />One of the big reasons we did not take more applications at Sabre to the web was the inability to manage in a user friendly, easy manner large sets of data in tables. The LiveGrid concept solves this problem. Think of it as GoogleGrids (aka, GoogleMaps)!<br /><br /><span style="font-weight: bold;">Pitch #2: To the Web World: Show Results in a Scrolling LiveGrid</span><br />I worked up a search engine example using the excellent <a href="http://developer.yahoo.net/">Yahoo Search Services API</a>. This illustrates what a search engine page could look like with the LiveGrid behavior. Its really quite fun to play with. I have demonstrated it to computer novices who are not easily enamored by technical stuff. Their response has been, "Why don't they already do it this way?"<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://static.flickr.com/29/58655484_03fa649bb5.jpg"><img style="margin: 0pt 10px 10px 0pt; width:400px; cursor: pointer" src="http://static.flickr.com/29/58655484_03fa649bb5.jpg" alt="" border="0" /></a><br />In the Yahoo Search example, we are bringing back live data from the search (usually around 35 rows for caching). Then as you scroll we are buffering as needed around you. The result is you can jump anywhere in the search data. Currently, the scroller is limited to 1000 results (the limit placed by the Yahoo Search API).<br /><br /><div style="clear: both;"><span style="font-weight: bold;">Pros</span></div><ul> <li>Familiarity. Anyone who has used a PC/Mac is familar with a scrollbar</li> <li>Ease of use. Just scroll it baby.</li> <li>Control is visible along the side, not hidden at the bottom</li> <li>Shows you how much data you have and where you are in relation</li> <li>With the tooltip and the results feedback it provides you something to watch and helps you to navigate while data loads.</li> <li>I don't have to configure how much I want to see on a page. Designers decide what works best and then you can just scroll through it.<br /></li> </ul> <span style="font-weight: bold;">Cons</span><span style="font-weight: bold;"></span><br /><ul> <li>The old problem of I have two scrollbars. One in the window around the results the other on the page itself. [Solutions: make scrollbar styled differently? Keep the complete scroller above the fold]</li> <li>If you have millions of results, you might not want to set the scroller extent to millions. The granularity of the scroll bar gets to be too small. However, it would be fascinating to actually scroll down to those last results. Anybody actually seen those? Do we even know if they exist ;-)</li> <li>Scroller extending below the page fold. Perhaps the best use is to keep the scrolled area completely above the fold. But hey, since you can scroll so fast through your data you won't miss the few extra rows of information below the fold (that you had to scroll to anyway.)</li> </ul><br /><span style="font-weight: bold;">Feedback Design Considerations<br /></span>We have experimented with several ideas to provide adequate feedback. During scrolling at certain points (when crossing a buffer boundary) there can be a slight delay. We plan on adding a twirling wheel (as in Firefox or Mac) just above the scrollbar and to the right of the status to indicate when the table is busy fetching data. Is this the right approach? The balance here is to not show it too often. It should be a lot like Dunstan Orchard's Live Search on his http://1976design.com/blog which waits to search when the user pauses so the number of updates is minimized.<br /><br />We added a tooltip (right now it does not fade away-- thats a bug -- but I have come to like it the way it is :-) that shows which record you are currently scrolling through. Also I dynamically update the status bar to show the range of records that are being scrolled through. I think both together give enough assurance that something is happening even with a slight delay in loading data. Certainly there are other ways to provide the feeling of scrolling while waiting slightly for data to flow in.<br /><br /><span style="font-weight: bold;">Your Feedback</span><br />Ultimately, its everyone in the application and web community that that will judge if the time is ripe for banishing many of the ubiquitous Prev/Next Page bars in favor of the scrollbar.<br />I am convinced this technique has important implications in the corporate application world and has the potential to change the way we interact with many of the search results, product listings and data grids we find on the web.<br /><br />We welcome your <a href="mailto:dev@rico.tigris.org">feedback</a>!<br /><br /><span style="font-weight: bold;">The Demos</span><br />You can find the two demos at:<br /><br /><a href="http://openrico.org/demos/livegrid">http://openrico.org/demos/livegrid</a><br /><br /><span style="font-weight:bold;">Note: This link is broken as of 2007. I will post an alternate location for the demo of the live search.</span> <strike>http://openrico.org/yahooSearch.page</strike><br /><br /><span style="font-weight: bold;">The Solution</span><br />It should be noted that besides the demos, we are providing open source code to make your tables behave in this manner. Try it, enjoy it and make your sites and applications better. Report bugs to us so that we can provide the best LiveGrid behavior possible.