Wednesday, May 11, 2011

Clean CSS with Server-Side Browser Detection (1/2)

Why?!
We all know that browser detection is bad, but let us look at some situations in which it may actually make sense; consider the following use cases:
  1. A specific (old?) browser needs some special CSS styles to make things look/work acceptably.
    For example, Firefox 3.0 needs a negative margin on an INPUT field, to correct a bug.
  2. Certain content should be highlighted to give a visual clue that this content appears most appropriate for the visitor's context, like browser and OS.
    For example, on a download page, you want to highlight a download for Mac OS X 10.6, because that is what the visitor appears to use.
Alternative Solutions
Typical solutions for these problems are:
  1. Use CSS hacks.
    Downsides: Unreliable, cumbersome and very hard to maintain.
  2. Use conditional comments.
    Downside: IE-only.
  3. Use feature detection. Various client-side frameworks support this, e.g., Modernizr or jQuery's $.support.
    Downsides: Only works when JavaScript is enabled, only on more recent, capable browsers and may impact performance negatively on less capable devices.
The Case for a Different Path
Excluding the first one, the mentioned solutions have their applications in modern web development. However, they certainly do not cover all ground.

An alternative solution for the specified use cases would be welcome if it would comply with all or most of the following requirements:
  1. Reliable, e.g. should automatically pick up new browser versions.
  2. Easy to apply.
  3. Easy to maintain, i.e., no hacks like html>/**/body {}.
  4. Generally applicable, i.e., not specific to IE.
  5. Supports Progressive Enhancement, i.e., does not require JavaScript.
  6. Supports older and less capable browsers as well as more recent and capable ones.
  7. High performance, no time lost detecting features client-side on a limited CPU.
Server-Side Detection
An alternative solution is to have a server-side algorithm add some CSS classes to an element on the HTML document. For example, an iPad agent string could result in the following BODY tag being generated:
<BODY class="Device-Tablet Browser-Safari BrowserEngine-WebKit BrowserOS-iOS">
Let's look at what such an approach has to bring to the table in terms of requirements coverage:
  1. Reliability: Fair/good. Newer browser versions will be picked up automatically or with an update of the detection algorithm.
  2. Easy to apply: Yes, definitely: BODY.BrowserOS-MacOS .MacDownload {border: 1px solid green;}
  3. Easy to maintain: Yes, you only deal with (readable!) CSS.
  4. Generally applicable: Yes, only limited by what user agents offer (of course) and what the algorithm detects; but the algorithm can be extended.
  5. Supports Progressive Enhancement: Yes, perfectly. Works without a single line of JavaScript.
  6. Supports older and less capable browsers: Yes, the detection is done server-side, independent of the browser.
  7. High performance: Yes, server-side algorithm should take less than 10 ms.

What's Next?
Does it sound like there is a valid use case for server-side browser detection? Let me know in the comments!

In a followup article, I will give some code samples.

Monday, December 27, 2010

TDD versus DDT

Test-driven development (TDD) is commonly accepted as a practical approach for producing high quality production code and high quality unit tests at virtually the same time.

This short article is about an alternative approach, called Documentation-driven testing (DDT). While it does not apply in all situation (which approach does?) I have used it effectively, although until now I've never taken the effort to describe how we did it.

This is how TDD works:
  1. Write a minimal test for the currently non-existent production code; no more than is needed to make the test fail; and not compiling is a form of failing.
  2. Make sure the test fails.
  3. Write the production code; program only what is needed to make the test succeed. And yes, take this literally, even if it appears stupid (e.g. returning a constant from a computation method).
  4. Make sure the test succeeds.
  5. Either go to step 1 or refactor the code and go to step 3.
Now, here is how DDT works:
  1. Write Javadoc API documentation for the production code, with a just an no-op implementation. The code should compile and the Javadoc API documentation should be generated.
  2. Write unit tests using only that documentation. This should be done by somebody other than the one that wrote the documentation.
  3. Make sure the test fails.
  4. Implement the production code.
  5. Make sure the test succeeds and, if not, find the source of the error (documentation, production code or test code).
  6. Go back to step 1 or refactor the code and go to step 5.
Now, what is the advantage of this approach? The main driver is the quality of the documentation, including the consistency with the production code. Note that you should already get a good review of the quality of that documentation in step 2, since a lack of understanding on the side of the unit test author should ring some bells.

It is clear this approach is not applicable everywhere, all the time. As is suggested in Uncle Bob's great book Clean Code, avoid comments except for public APIs. There it may be an approach that works for your team; if it works for you (or not) do let me know!

Tuesday, October 19, 2010

Bizarre: Site redirecting to about:blank

The site www.vbtelecom.nl is redirecting to about:blank. How bizarre! And it works in both Firefox 3.6.10 and Safari 5.0.2:
$ telnet www.vbtelecom.nl 80
Trying 77.95.248.16...
Connected to redirect.server.eu.
Escape character is '^]'.
GET / HTTP/1.1
Host: www.vbtelecom.nl

HTTP/1.1 302 Found
Date: Tue, 19 Oct 2010 19:49:19 GMT
Server: Apache/2.2.9 (Debian) PHP/5.2.6-1+lenny6 with Suhosin-Patch
X-Powered-By: PHP/5.2.6-1+lenny6
Location: about:blank
Vary: Accept-Encoding
Content-Length: 0
Content-Type: text/html

Connection closed by foreign host.

Friday, April 30, 2010

XINS lacking IPv6 support

If you get the following error while running a XINS-based application:
org.xins.common.text.ParseException: The string "0:0:0:0:0:0:0:1%0" is not a valid IP address.

then you found out the hard way that XINS lacks IPv6 support. This applies to all current versions of XINS: 2.2, 2.3-beta1 and 3.0-beta1.

To work around this, set the system property java.net.preferIPv4Stack to true.

Wednesday, April 28, 2010

XINS 3.0-beta1 released

Just now I tagged 3.0-beta1 of my experimental XINS fork (which is not so experimental from the perspective that it has been running in production-sites since 2007).

Source and related stuff available from GitHub:
For a complete download, get this ZIP package:
Recent changes include:
  • Ant 1.8.0 is now properly supported
  • Logdoc definitions are now validated using XSDs (Logdoc 0.3 included)
  • Default runtime config file reload interval is now 5 instead of 60 seconds.
  • Not initializing logging subsystem (Log4J) if system property "org.xins.server.logging.init" is set to "false".
  • Not setting context IDs (in Log4J terminology: NDC.push() calls) if property "org.xins.server.contextID.push" is set to false (system, bootstrap or runtime property).

Saturday, April 17, 2010

The importance of contract-first development

In enterprise IT projects, software services are typically consumed by other teams or even organisations than those who programmed those services. For example, a Billing system can be instructed by a User Profile system to allocate billing details for a new user profile. Or a Web Frontend system can ask a Session Management system for the data pertaining to a certain session ID.

This is typical service-orientation.

Expectation management
A challenge in such a service-oriented approach is making sure that all involved parties – and that can be or become a lot more than just 2 – have the same expectations from the service. In my experience this is the number one source of frustrations and issues.

Before a service is built or extended, the parties involved at that point in time typically reach agreement what the new functionality should do and how it should be accessed. It is then quite common to have those teams work in parallel, because time-to-market is key.

But even if the teams do not work in parallel, when the service provider delivers functionality for testing and integration, expectations from the other parties can be very hard to meet. For example, service consumers may have expected different formats or may have expected necessary transformations would be done by the service provider, while the service provider is pushing the requirement to the callers.

When this happens, project deadlines and stress levels are at stake.

And the more consumers and the more time it takes for the service provider to deliver a first version, the higher the risk.

Working with contracts
The common solution to this is to work with a contract that defines the agreements between the provider(s) and the consumer(s). Such contracts vary from informal (e.g. Word-documents lacking strict interface definitions) to formal (WSDL with very strict constraints).

Such contracts can be written at the start or they can be generated from existing code.

Generating a contract from existing code is generally a bad idea. The contract is then focused on the implementation, which is subject to change. Then if the implementation changes, what happens to the interface? If it should also change, this violates the common principle of Separation of Concerns. And it is clearly the wrong way around, the contract is the agreement, not the implementation chosen by the provider.

Another issue with contract generation is that the contract can be very much focused on the implementation, including specific data types tailored towards an implementation within a certain platform or programming language. This has an impact on the callers and, in the long term, also on the consumer, since the latter may be forced to stick with legacy implementation choices.

An implementation-independent contract
A first step towards a complete solution to this challenge is to author a contract that is implementation-independent. Such a contract can be agreed upon from the start by all involved parties. Real separation of concerns: the contract is the common agreement, while providers and consumers are free to make their own choices within the bounds of the contract.

This may appear a complete solution.

However, in practice, contracts change, possibly even during the initial implementation project. So what should be done in such a case? Is the contract still leading or will the contract by bypassed by reality?

Based on my experience in professional environments I dare say that if the contract can be violated, it will be. With impact on stability/reliability and on stress levels. In the battle between contract-orientation and project deadlines, the former is easily sacrificed.

Contract-first development
The only way to really guarantee that you can keep the promise of an independent contract is to integrate your contract-orientation in your development process, making the implementation(s) dependant on the contract, preferrably on both sides of the contract (providers as well as consumers).

One way to achiebe this is to make your code link to the contract somehow, for example by having a URL to your WSDL files in all implementations, making sure that failures to adhere to the contract result in errors.

Validation at runtime
To avoid the contract being bypassed, implementations should check all service invocations to make sure the contract is respected.

This can be achieved by generating server- and client-side code from the contract. Having access to the contract (in some form) the implementation then checks all aspects of the invocation. Not only the  preconditions (consumer requests), but also postconditions (provider responses). If either a consumer or a provider violates to the contract, that must be treated as a fatal violation. Only then your contract will actually be leading.

Validation on the server versus on the client
So if you want to validate your contract, where should you do it: on the client (consumer) or on the server (provider)?

The best approach to this is to do it in both locations, for good reasons:
  1. Separation of concerns: both ends should assume their responsibility, which is to adhere to the contract and consider violations fatal. Neither end can be sure the other end will validate the contract, so they should not assume that.
  2. The provider and the consumer may have access to different versions of the contract. Validation on both ends will likely catch any incompatible differences in the contract versions.
Your contract is your treasure!
Now when you have accomplished a true contract-oriented development process, you will find the contract is your gold mine. The contract is typically orders of magnitude simpler than implementations, making it easy to read and change. But since it describes an extremely important integration point so well (concise) and so correctly, it is easy to reuse it for all sorts of of purposes. For example:
  • generation of documentation, e.g. Word-documents or web pages, etc.
  • generation of forms or programs for manual use (including human testing)
  • generation of unit-, performance- and load-tests
  • generation of IDE projects, such as Visual Studio, Eclipse, NetBeans, XCode, etc.
To enable certain possibilities, the formal part of the contract needs an additional informal part. The informal part can include examples and human-readable descriptions. They can also help humans understand the contract and its purpose faster and more easily.

Examples of contract-orientation
While WSDL is an example of a contract definition language for web services, it does not provides a full contract-orientation process, requiring the contract to be held.

The only fully contract-oriented projects I am aware of are those I initiated: XINS and Logdoc. The first (XINS) is an open-source web services framework (that generates server- and client-side Java code), while the latter (Logdoc) implements contracts for logging, again including a Java-implementation.

Do you have experience with contract-orientation? Do you know of other projects that put contract-orientation in the heart of their approach? Please let me know in the comments!

Tuesday, March 30, 2010

ATG 9.1 on Mac OS X 10.6

Using the instructions from Ronald Pulleman (which focus on ATG 2007.1), I tried installing ATG 9.1 on Mac OS X 10.6.2. Below are the instructions for reproducing what worked for me.

Make sure you know what is officially supported by ATG, since Mac OS X is not. Also make sure you have the Installation Guide handy, part of the ATG 9.1 documentation set.

Below I refer to the .profile file for adjusting environment variables. Make sure you apply these environment variable changes each time. If you do not use bash, then replace .profile with whatever your shell reads on startup.

Install and configure MySQL

  • Have MySQL installed; I picked MySQL 5.1.45 (64-bit), but 5.1.32 would have been supported.
  • Add this your .profile:
    export MYSQL_HOME=/usr/local/mysql-5.1.45-osx10.6-x86_64
    export PATH=$MYSQL_HOME/bin:$PATH
  • In your System Preferences application, start MySQL and make it start automatically at boot.
  • Change the password of the root user, for obvious reasons:
    mysqladmin -p -u root PASSWORD newpass
  • Make MySQL use the InnoDB engine by default, this is required for ATG: Create the file /etc/mysql/my.cnf with the following contents:
    [mysqld]
    default-storage-engine=innodb
    sql-mode=TRADITIONAL

Install and configure JBoss

  • Install JBoss; I used JBoss 4.2.3-GA (community edition), while the (paid) JBoss EAP 4.2.0 CP05 would be the ATG-supported version.
  • Add this to your .profile:
    export JBOSS_HOME=$HOME/jboss-4.2.3.GA
    export PATH=$JBOSS_HOME/bin:$PATH
  • Modify the JVM arguments in the startup script, if you want to comply with the ATG recommendations for JVM parameters. Edit the bin/run.sh file and modify the JAVA_OPTS setting as follows:
    JAVA_OPTS="-server -Xms2048m -Xmx3072m -XX:MaxPermSize=768m -XX:MaxNewSize=768m -Dsun.rmi.dgc.server.gcInterval=3600000 -Dsun.rmi.client.gcInterval=3600000"

Integrate MySQL with JBoss

  • Download the MySQL JDBC driver Connector/J, I got version 5.1.12.
  • Copy the Connector/J JAR file to the server/atg/lib directory under the JBoss home directory:
    cp ~/Downloads/mysql-connector-java-5.1.12/mysql-connector-java-5.1.12-bin.jar $JBOSS_HOME/server/atg/lib/

Install and configure ATG

  • Download ATG 9.1 from the ATG support site, this is a JAR file.
  • Unpack it and then execute the contained .bin file:
    unzip ATG9.1.jar
    sh ATG9.1.bin
    Choose JBoss as the application server. I suggest you make sure there is no space in the name of the destination director (I chose ~/atg-9.1).
  • Put your license keys in the home/localconfig/ directory (below the ATG installation directory).
  • Create the file home/bin/dasEnv.sh with just one line in it:
    export ATGJRE=/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Commands/java
  • Create the file home/localconfig/atg/dynamo/service/jdbc/JTDataSource.properties with the following contents:
    JNDIName=java:/MySqlDS
    $class=atg.nucleus.JNDIReference

Initialize the ATG database

  • Under the JBoss directory, create the file server/atg/deploy/mysql-ds.xml with the following contents:
    <datasources>
      <local-tx-datasource>
        <jndi-name>MySqlDS</jndi-name>
        <connection-url>jdbc:mysql://127.0.0.1:3306/atg000</connection-url>
        <driver-class>com.mysql.jdbc.Driver</driver-class>
        <user-name>atg</user-name>
        <password>atgpass</password>
        <exception-sorter-class-name>org.jboss.resource.adapter.jdbc.vendor.MySQLExceptionSorter</exception-sorter-class-name>
        <valid-connection-checker-class-name>org.jboss.resource.adapter.jdbc.vendor.MySQLValidConnectionChecker</valid-connection-checker-class-name>
        <metadata>
          <type-mapping>mySQL</type-mapping>
        </metadata>
      </local-tx-datasource>
    </datasources>
  • Log in to the MySQL command line client mysql:
    mysql -p -u root
  • Create a database user for ATG:
    USE mysql
    INSERT INTO user (host, user, password) VALUES ('%', 'atg', PASSWORD('atgpass'));
  • Create a database for ATG:
    CREATE DATABASE atg000;
  • Allow the atg user all access privileges on the atg000 database:
    GRANT ALL PRIVILEGES ON atg000.* TO atg;
    FLUSH PRIVILEGES;
  • Exit the mysql client (you can use Ctrl-D).
  • Create the ATG data structures in the database by executing the DDL scripts on MySQL (this typically takes less than a minute):
    cat DAS/sql/install/mysql/das_ddl.sql DPS/sql/install/mysql/dps_ddl.sql DSS/sql/install/mysql/dss_ddl.sql DCS/sql/install/mysql/dcs_ddl.sql | mysql -D atg000 -u atg -patgpass

Run ATG

  • Deploy an ATG application to JBoss.
  • Start JBoss: Go to the JBoss directory and execute:
    bin/run.sh -c atg
  • Using a browser visit http://127.0.0.1:8080/dyn/admin/ to see the DynAdmin tool.

Wednesday, February 17, 2010

Keep your (webmail?) session open

If you have a website that keeps expiring your session after a while, and you want to work around that, then try out the SessionAlive extension for Firefox. I find it extremely useful, especially for webmail (such as Microsoft Outlook Web Access).

The extension works OK in Firefox 3.6.

Tuesday, February 16, 2010

Experimental XINS 3.0-alpha4 released

Just now I've tagged 3.0-alpha4 on my experimental XINS-fork. Changes since 3.0-alpha3:
  • newer version of Logdoc, which no longer requires a DTD to be set
  • JMX port can be specified on the command line, e.g. "xins run-abc -Djmx.port=2090"
  • APIs using Logdoc actually have proper code (Log.java and TranslationBundle*.java files) generated
  • the 3 Library classes (in common, client and server) actually return the version of the library (instead of null) based on the meta information available from the JAR manifest
  • fixed client-side transaction logging, it actually works now
  • the _GetVersion meta-function now also returns the current Logdoc version
  • the _GetStatistics meta-function now supports a "targetFunction" parameter, to reduce the output to the statistics of one function only
  • the _GetStatistics meta-function now also returns the transaction ID for each call that information is provided for
  • _GetStatistics not only returns the average, but variance as well (compute the standard deviation from that by taking the square of it)
  • fixed and improved the unit tests
Go check it out at http://github.com/znerd/xins/tree/xins-3.0-alpha4.