What is wrong in Fatwire/WCS 3: Mixing HTML and Code
It is hard to believe that in 2012 we still we have problems of mixing html and code. Those problems  are supposed to have been solved since many years, but unfortunately they are still around.

In an ideal world, a web designer is also able to code. In a world even better, a web designer is also able to write jsp templates and knows fatwire tags.

In the real world, web designers, while often knowledgeable of  javascript, have usually no clue of JSP nor of specific Fatwire coding, so they produce only a mockup made of client-side HTML, CSS and Javascript. They leave to  "server side" developers the task of using that HTML/CSS/Javascript to create a content manageable website.

The problem with Fatwire/WCS is that to render the mockup code, you have to add Fatwire coding to it. So , because WCS is still a JSP based system,  you have to add  logic to extract the content model and put it in place in the HTML mockup. That would be fine if... it were to be done  only once.

But in reality, HTML code undergoes a number of iterations. Web designer will update the mockup and return it to server side developers. Those developers will  have to use the new HTML to update their templates code, that is now an heavily modified version, with added a lot logic in Java and JSP tags.

This is a big problem, because it is not easy. The usual process is figuring out what changed in the mockup  from the latest version used to build templates,  then go through the code to apply those modifications. Hoping that nothing will break.

When the HTML is heavily modified, starting from scratch is not unusual. In short, the process of updating templates when the HTML mockup changes is a real pain in the ass.

What is really needed.
The principle of separating presentation code (HTML) from logic is almost as old as the web itself. There are millions of solutions arounds for this, yet those solutions did not yet reach WCS in his core. We strongly need to implement for real this separations.

Furthermore, since HTML mockups are going to change ofter, we should be able to leave the mockup in his original form, in order to be easily updatable. Rendering logic should be applicable directly to HTML in his original form, without modifications.



Filed under: Tutorial No Comments
09
Jan
2013
What is wrong in Fatwire/WCS development 2: Deployment Hell
One "feature" of WCS, being a CMS, is deployment of code done in the same way as content. Code is treated as content and managed in the same way. We will see that this fact can create many problems.

A content editor doing is job changes some content, and then he approves it for publishing. WCS is smart enough to detect dependent content and requires the approval of related content to publish it in single publishing session.

This is great for a web site, where you only have to update a single content asset to update all the web pages referring to that content in the web site. Furthermore the publishing process is smart enough to invalidate only parts of the cache affected by the  changed content.

Developers should work in the same way: in the development server, a developer  can change  the template code, then approve it and finally publish it. Code should then go from development servers to staging servers and finally to live servers.

Let's put aside for now that having a single development server for multiple developers is a problem in itself (I will say more of this later), let's give a look at what developers really do and why this way of developing the code does not work as good as it should.

How developers REALLY develop...
There is a great variation in development procedures. The more common, even if now there better tools are available, many are still using the aging ContentServer Explorer  (now Sites Explorer) editing directly JSP code stored in the  ElementCatalog table.

Unfortunately, when you edit a JSP, the associated Template or CSElement is not aware that you changed the code with CSExplorer. So to make sure the "code publishing" mechanism work, you have to manually edit and save the   Template or CSElement corresponding to the JSP you edited, then approve it and finally publish it.

Being a manual process, way too often happens that someone forget either the edit/save or the approval of a changed template.

Also the propagation of the code from staging to live requires a re-approval of the templates. Although you can theoretically could just do a bulk approve, many people are scared of republishing everything. So what usually happens is that all the changed templates are manually approved and published, using manually kept release notes.

Since usually who deployed templates from development to staging is a different person from whom developed them, a floating document with the list of the changed element, or worse a flow of random email is used to propagate those informations.

At some point someone makes a mistake, forget to approve a template, distribute a list with the wrong templates ... and  problems not existing in development starts to appear in staging or in production, randomly.

When different developers are involved, or there is a turnaround in the editorial team, it happens way to often that you no more know what is deployed in which server. I have seen people periodically spending days comparing each template in different server just to figure out what went wrong and the origin of a bug.

But wait... there is more
Actually things can go much worse than this.

Another problem happens  very often when developers are forced to develop on a system disconnected from the staging/deliver chain.

This may happen for many reasons, the more common is some brain-damaged security policy, but there can are other more practical reasons, for example: "the connection from UK to India is too slow and we had to deploy a local development server".

The current solution to this problem is CSDT but to be honest, it is not  yet very widely used. People are very creative in solving the problem of distributing their development work. Some uses catalog mover, but I have seen people distributing their work as a database dump and even manually copying and pasting the code  in the Fatwire Advance Interface.

Needless to say, this is aggravating the deployment hell already described in the previous paragraph.

But the worse situation, that I have seen too, is when developers are developing in their development server, then some other people fix some issues (usually in HTML) editing directly templates in staging, and at the same time some urgent issues are also fixed manually editing templates  directly on the live server. The result as you can imagine is a total unmanageable mess. And unfortunately, even is is an extreme case, it happens.

What you really need
 The whole idea of deploying code as a collection of separate elements singularly deployable is wrong, and all this is originated because code is treated as content.
Code is inherently different from content. It has a different structure, a different  development process, different editing tools. More important, code must be always deployed always as a single unit, ideally as a single file, easily trackable and recognisable. You should immediately know that you have for example version 1.3.14 on production, and bug report and fixes must refer to a specific version number.
So the fact that Fatwire/WCS allows code to be treated as separate entities independently deployable is a weakness.
Java has a concept of  deployment unit:  it is called the "jar" file. Fatwire/WCS is one of the few Java places where code is not deployed in jars, but it is instead delivered as separate templates deployed through publishing.

What is really needed is that all the code for a site can be distribuited as single JAR fil, that can be easily deployed, tracked, compared, distribuited and versioned.

All the deployment hell I described would go away if instead of having a bunch of files, you have a jar. The jar can be built by developers, tested separately with bug report and fixed referring to a specific build, delivered to destination and deployed just copying the file and eventually running some schema update procedure.

Jars have a shortcoming, of course. Usually they require the restart of the application server to be recognized. This in a live environment is not usually acceptable. Nonetheless,  it is not always true that deploying a new jar requires the application server restart. There are plenty of hot-reloading  Java systems. Just to mention one, hot reloading of jars in JBoss. So it is possible a system where a site is deployed in jars that are deployed without restarting the application server (and indeed, I already implemented such a system).

I will continue to list WCS/Fatwire development problems in the next few posts before introducing my solution to those problems. Stay tuned.



Filed under: Features, Tutorial No Comments
07
Jan
2013
What is wrong in Fatwire/WCS development 1: JSP are evil
I am close to release a new open source project that  tries to fix  many of the common problems in Fatwire and WCS development.

But before the release, I want to  discuss those problems, since fixing them  is the main motivations underlying my development effort.

Actually, many of the features of the framework I am doing cannot be understood unless you know which real world problems they try to solve.

This is not a rant.  I also have a solution to those problems and it will follow in the next few weeks.

JSP are evil
The biggest issue in Fatwire/WCS in my view is that you are stuck with JSP (or worse, with the obsolete, very limited and almost undocumented "Fatwire XML").

JSP are designed only to be a quick way of rendering a dynamic HTML page using Java code. Something that is 95% html and only a little bit of code. They are no way  meant in no way to be a complete tool for doing generic Java coding.

They provide the flexibility of immediate availability (since they are recompiled and reloaded on the fly), and they provide the "markup first" approach that is useful when you need to render a lot of HTML and only a small part need to be coded and rendered dynamically.

Indeed this JSP  flexibility comes with a lot of limitation, while the coding in Fatwire development tend to be pretty complex, so complex that you end up writing big JSP so full of code that  may you have difficulties finding the HTML.

Here some of the limitations of JSP. First and before all, because a JSP is  a single method of a single class, you generally are not supposed to define methods and classes. You actually can define classes and methods, using the "<%!" syntax.

Hovewer, since it is not the way  JSP are supposed to be used, you cannot reuse any defined  method in another JSP . This is even worse when you also define a class. You can create classes inside a JSP but you cannot reuse that class in another JSP.

The only way of having code shared between different JSP is to create a separate jar with all the code and deploy in the application server Coding such a jar is relatively awkward, because you have to build it, deploy and restart the application server for every change. So usually this is done rarely.

For this reason, building libraries of code is not normally done in Java (as it should). Instead, the common practice is to create a library of "elements" called by JSP.

The problem is that a CSElement is not really meant to be a library doing complex things, it is meant as a medium to generate repeatable rendering of common elements.

The semantics of calling  CSElements, and using them as a library is, frankly, disgusting. There is no way return value from a CSElement , so you normally use a global environment and  side effects (altering the value of a variable value in a shared environment).

The JSP "language" does not offer any enforcement to using a CSElement as a library call, so everything is left to convention. You need to document clearly which variables are changed to see what is returned. This practice is error prone, hard to read and even harder to maintain.

Also it happen often that a CSElement "pollutes" the calling enviroment, so you need to use pushvar and popvar to preserve the environment. And this makes the whole procedure even more disgusting, unreadable and produces really bad code, where a lot of complexity is there just to move around variables, protect them and read side effects.

Last but not least, the invoking CSElements is really verbose and typing an invocation method is often so long that you copy and paste code. Long code doing little is also very hard to read.

How to solve JSP problems
1. What really is needed is a way to code in full, plain, clean Java, not in JSP. Classes offers all the power needed to build complex programs, and a JSP are a simplified AND limited form of Java classes. Using Java will immediately give you the ability to write methods and classes, keeping them totally reusable.

2. However you need to retain the ability of seeing immediately the result of a change because restarting the application server at each change is usually not really practical:  it is so slow that developers to avoid it will prefer to push tons of code in the JSP. A solution could be "JRebel" but since it is expensive for some teams (most notably indian teams) buying it can be a problem. So that solution should be cheap.

3. Coding in Java you will also want to avoid ton of out.println("...") filled of "\" to escape quotes just to generate HTML. Probably you may not not the "HTML first" approach of JSP  and you may prefer the "code first" of Java, but you still want an easy way of generating HTML.

4. Last but not least, JSP offers some well defined tag libraries for rendering the content model. Since the equivalent Fatwire Java API (the Asset API) is not even close to the quality of tag libraries, you need some  efficient way of invoking tag directly from Java code.

We have not finished, yet.
Actually we have just started, a lot more will come. So please wait, there are more problems to  discuss in the next posts. And the solution will follow, it is a promise. So stay tuned.


Filed under: Tutorial No Comments
10
Jun
2012
The new WCS User Interface
The new  Oracle-branded WecCenter Sites v11g is not a revolutionary product: it is still recognizable as the old, good ContentServer (formerly Fatwire, Divine and OpenMarket) we already know.  Nonetheless, in this new incarnation it looks like a big jump forward.

This release is probably defined more from what has been removed than from what has been added. Most of  the work done  on WCS looks to be a great effort to simplify, make uniform and consistent,  overall improve the user experience of the product. The first and more evident feature gone is the dash interface (but not its functionalities). It is also gone the insite interface,  but again,  insite editing is now even more powerful.

There is no more the dualism of 3 competing user interface (dash, advanced and insite), where you can do more or less the same thing in different incarnations. Instead, there is now one unified UI. It acts like the old Advanced interface when it comes to manage "advanced" content assets (mostly used by developers and webmasters) . The UI also acts like the dash interface to browse and edit content, but seamlessly integrates  insite editing to preview and edit content at the same time.

Let's do a walkthrough with screenshots to  understand better.

The Administrative interface
Let's start from the administrative (the old avanced) interface. It is now probably  going to be the less used but we start from it because it is familiar to the vast majority of current users.

On the top of the screen there is now a toolbar allowing to select the interface (now there are 2, the administrative and the contributor). If we select the administrative interface, as below:





It is now possible to create and search assets, but, important new, ONLY administrative assets: templates, cselements, attributes, definitioins and so on. It is no more possible create and edit content assets from this interface.

Creating an asset is still done in the old way. For example clicking on a start menu link to create a new template:





then the usual form is shown:





nothing new under the sun regarding the administrative assets (except a css restyling). The interesting parts comes now.

 The Contributor interface
When we select the new Contributor Interface things start to became new (and interesting):



Creating a new element now requires you select it from the menu. The new  action  now appears in a pull down menu:





Content can be edited now in a form, so in a sense it is still possible to do form oriented content editing. See below and please note the toolbar of saving, approving , previewing and deleting.




However, the interesting part is the switch to enable a visually oriented content editing. Just click on the switch and the form becames an (editable) preview.

Here the content editing is pretty rich. You can drag and drop an image to insert it  in an area of the page, click on a text field to do inline editing, click on a date area to display a date picker and so on.

Definitely, now the content editing integrates the 2 different interfaces (dash and insite) but it has became much easier to understand. Overall the user experience is way simpler and straightforward. Furthermore, a lot of duplications are removed, while keeping and even improving the features already available.
Definitely a sign of product maturation worth to be now the Oracle CMS. Because the software has been developed before the acquisition, it is worth to say: excellent job, Fatwire!