Point-to-point SOA

p, li { white-space: pre-wrap; }

Service Oriented Architectures realizations are coupling departments within large organizations. Web-Services are being developed without compliance or guidance from any actionable enterprise architecture design. Without a full soup-to-nuts solution architecture template available to teams, well-intentioned developers are unfortunately creating a web of point-to-point systems architectures with SOA tools and technologies.

Vendors, marketing, tooling and technology isn't helping integration architectures evolve and arrive at the optimal quality attribute tradeoffs. Most quality attributes, like loose coupling, high cohesion, performance, scalability etc are skewing negatively.

It cannot be stressed enough that SOA is more dangerous with out a proper design in place, than no SOA at all. WSDL, SOAP, XSD, REST etc are still not properly understood or realized appropriately in many organizations. Basic systems architectures are unable to evolve due to a significant lack of skills, organizational constraints, poor processes, personnel and such.

A Combination Algorithm

The problem: Write an algorithm that can provide combinations of a given word. The combinations need not contain a new word with the same letters.

Input: ABCDE

OUTPUT: ABCDE, ABCD, BCDE, ABC, BCD, CDE, AB, BC, CD, DE etc.

There are several algorithms out there that describe how to create effective combinations, SEPA does permutations, bubble sort and nested loops etc.

After spending some time looking for what’s been done - the one I liked the most was as follows: [0,0,0], [0,0,1], [0,1,1] [1,1,1]: null, A, AB ABC etc.

Incrementing decimals, start with zero, get a binary representation and map it to the array positions. This will give the combinations in constant time. A very efficient algorithm.

   /**
    * Take a word and return a collection of combinations
    * @param sortedWord
    * @return
    */
   public Collection<String> generateCombinations(String sortedWord)
   {
       Collection<String> c=new HashSet<String>();
       char[] broken=sortedWord.toCharArray();
       int combinationLength=new Double(Math.pow(2L, (new Long(sortedWord.length())).longValue())).intValue();
    //  System.out.println("Combination len="+combinationLength);
       for(int loop=combinationLength-1;loop>0;loop--)
       {
       String binary=fillWithZeros(Long.toBinaryString(loop),sortedWord.length());
       char[] bins=binary.toCharArray();
       StringBuffer wb=new StringBuffer();
       for(int a=0;a<bins.length;a++)
       {
           Integer binValue=new Integer(""+bins[a]);
           if(binValue.intValue()==1)
           {
            wb.append(broken[a]);  
           }
       }
       c.add(wb.toString());
       }

 

While there is a nested loop, the loops can be replaced with other techniques.

This is a very high performance combination creation technique. Comments/Improvements are welcome.

Tomcat, Struts, Tiles and Hibernate Migration

I had an old ‘home’ project that was based on the following technologies:

(1) Tomcat

(2) Hibernate

(3) MySQL

(4) Struts 1x

(5) Tiles 1x

I had developed this in 2004-2005 for a user-experience experiment I had designed.

I just decided to upgrade the technologies to :

(1) Geronimo

(2) OpenJPA

(3) MySQL

(4) Struts 2

(5) Tiles 2.2x

Why ? Mainly to experience what a migration path might be.

As I go through the effort – it turns out that there is no easy migration path. OpenJPA is vastly different from Hibernate. Struts 1x and Struts 2x have very little in common. It is a full rewrite. Tiles 2x has the most common features deprecated, and it too is a rewrite – given the jsp pages need to be updated. The latest version of Tiles is incompatible with the latest version of Struts (as of May 2010).

 

java.lang.NullPointerException
	org.apache.tiles.definition.UnresolvingLocaleDefinitionsFactory.getDefinition(UnresolvingLocaleDefinitionsFactory.java:102)

 

WARN  [TilesContainerFactory] DEPRECATION WARNING! You are using parameter-based initialization, that is no longer supported! Please see docs: http://tiles.apache.org/framework/tutorial/configuration.html

You’re better off using an older version of Tiles as long as you don’t care about Freemarker etc and are simply going to use JSPs on the view.

All in all, there is no migration it is a full rework/refactor/rewrite and is pretty significant and directly correlated to the size of the enterprise app. This information is good to have for those looking to ‘migrate’ legacy JEE apps.

Debate over Domain Agnostic Connections

I recently introduced after a proof of concept and technology Service Integration Bus to enable Event Driven Architecture style integration across the commercial enterprise.

The first team implemented it exactly as the POC specified. The second team that was responsible to be the durable subscriber raised concerns about Domain Agnostic Connection factories.

The claim was that using domain Specific Connection Factories makes life easier. The basic difference between the two is that one uses for example TopicConnectionFactory to create a TopicConnection, whereas the domain independent connection factory lets you look up a ConnectionFactory to create either a TopicConnection or a QueueConnection. This lets you get flexibility in the app and is basically a new feature in JMS specs.

Anyway the second team was up in arms about using domain agnostic connection factories. “Why is architecture recommending this? It’s making life difficult for us!”. Of course the first thing you learn in software architecture school is :make life hard for the development community. The tone and attitude changed at the end of this debate – architecture was a friend and developers happily followed the direction. How ? Surprised me. This blog entry is probably a note to myself.

How do you convince developers to do the right thing, ‘change’ and adopt a new way of doing things? Sift through words and select the right bytes. Software architects must recognize ‘code’ in spoken language. Understanding the subtle hints and uncovering the real issues. This is the key. Sounds like management skills? Yep, a little bit.

This is not science and they don’t teach you this in college. Negotiation and arguments must be approached uniquely each time based on context. When this ‘senior’ software dev was lamenting about how hard his life is going to get because architecture decided – he was starting getting an audience that did not understand the topic. He kept saying that we should not use Domain Agnostic Connection factories, I needed to break it down quickly and stop the nonsense. Usually when there is resistance to change, it is mostly because this new change has not really sunk in mentally. Addressing concerns objectively is the first step.  Use technical facts, lay it down straight. This needs to be the corner stop of every architect and a must-have skill. If you don’t know why you’re recommending something, just don’t. If you do -  SEI’s ATEC skills comes in pretty handy here.

In this particular case, the facts were that the recommendation had clear advantages. However, this did not seem to satisfy the senior developer. After some social digging it turns out that the fact that the architectural decision to use Domain Agnostic Connection Factories was made without the sr developer in question being involved was a contributing factor to the resistance. This little tit-bit of information was gathered from his peer. I opined that this was the key to his constant lamenting. With this hypothesis I formulated the next steps. Now, one has to be careful because this is the type of situation when people start looking backward and not forward. Even in hindsight, it was not possible to predict this engineer’s interest in the topic. It was explained that these decisions were made along with a host of other decisions through a Proof Of Concept and technology evaluation. This was reviewed and approved by other senior stakeholders. It was documented on the wiki and the link was sent via email. I worked with the technical person senior to him in his team – and explained the matter in technical terms. He heard from him and me, saw that others were on the same page too. A few more emails with links to document and an encouraging note – things started to look better. The attention he received probably made him realize that it wasn’t a purposeful exclusion on that decision but the natural process of technology selection by a whole host of senior technologists. The technical reason for using domain agnostic connection factories is sound and makes sense for advanced JMS systems that don’t want to have 2 connection factories depending on what messaging resource they are connecting to.

He replied back saying that he would follow this direction. It turns out that the direction was finally adopted. So what is the moral of the story ? Sometimes it matters how you make others feel about technology regardless of what the technology does for them.

Are DSLs simply XML Hell?

DSL : what is it ? You have used a Domain Specific Language. Trust me. Demystify DSL: it simply a narrowed and specific semantic problem that solves a narrow set of problems.

Model-View-Controller (MVC) is a common pattern for Web development, Apache Struts has come to the rescue in the past. Each Struts application uses a configuration file (XML). The language (dialect) of the XML specifies exactly how the MVC pattern would be implemented for the specific implementation this is DSL.

Object Relational mapping was a common problem made worse by general purpose languages and APIs(e.g. Java and JDBC), along came the a natural ORM (e.g. Hibernate) with specific configuration files that mapped POJOs to Tables. The configuration file (XML) supported a dialect (ORM DSL), of course it was not called DSL back then they were simply configuration files (aka XML Hell).

DSLs don’t necessarily mean XML. For example, running rules in Java was ‘simplified’ by a Rules Engine implementation e.g. Jess. This used a DSL - Jess 7 has its own declarative XML rule language called "JessML" (a DSL).

What is the advice for people who are trying to learn DSLs and master them ? Don’t. Try not to master DSLs instead focus on the domain of the problem, understand the software architecture and how it fits, understand what the pattern of problems are, figure out how best to apply design patterns that solve them. Understand the pain of the domain (e.g. web-site design, object-relational mapping framework, rules etc), understand the inefficiencies, then focus on an over all design within the domain that solves most of the problems. Finally, look at the DSL as a tool to solve the problems in a specific efficient way. Then you will see there is no XML but simply a graceful solution.

EJB 3.1 (Asynchronous Beans)

Enterprise Java Beans specifications have finally matured to a point where it may actually help with performance: container-managed Asynchronous beans.

You can now specify a Session Bean (and/or a method) to be asynchronous. End-user perceived wait times can be significantly reduced with an asynchronous programming model that doesn’t force the complexity of AJAX, MoM or application-managed server threads.
With EJB 3.1 when you call the method from your client, the container will take the request and return immediately. Then it will dispatch the request to a bean instance within the pool of SLSBs.
The return type for async methods is either void or Future. The Future interface provides methods to check whether the operation has completed, gets the result – with options for a timeout and also cancel the operation. A concrete implementation of Future is AsyncResult which is available in the javax.ejb package.
Stateless EJBs may choose to return the AsyncResult object with any type returned. Transaction propagation is simple &  limited - REQUIRED=REQUIRES_NEW attributes. If your client is already in a transaction, or your beans are in a transaction, the async method is called in a totally new transaction with no propagation (as in synchronous methods). This makes a lot of sense, because the entire purpose of asynchronous calls are to decouple a process orchestration to speed up things. However, it does complicate compensation and atomicity. Software architects need to trade-off between transaction demarcation and generally ACID properties across various calls. Any application exception that gets thrown by the bean for a void return type cannot be handled by the client. If your bean can encounter an exception and your client cares about handling it, then return a Future<V> form the bean. The client must handle the exception on a Future.get and getCause from ExecutionException.
An entire bean (itself or via it’s super class) can be marked as @Asynchronous. Sometimes it makes sense to expose the same business function as a synchronous and an asynchronous interface, using the @Asynchronous tag can provide the cohesion with a decoupled interaction model. This opens the door to multiple use-cases that need a simple way to conduct asynchronous calls without the complexity of MDBs or managing application owned threads. EJB 3.1 has built upon the POJO model for EJBs with an excellent way to realize asynchronous server-side models.

Google Chrome @ 5% (almost)

Google Chrome is now the 3rd most popular browser in the world.

According to recent reports here and here, Google's Chrome has overtaken Apple's Safari in the browser wars. While still at less than 5% chrome is no match for Microsoft's Internet Explorer which is slightly above 62%. Chrome is gaining strength quickly mostly due to two quality attributes: speed and reliability. Chrome is noticeably faster than Opera, IE, Firefox and Safari. It's reliability is unsurpassed due to its multi-process architecture.

Coming in at #3 just months after it's GA release is commendable. With new features like bookmark sync and support for themes and plug-ins it will rival the # 2 spot in a couple years, eating away Microsoft's and Mozilla's share.

SSH and Strawberry Perl

Strawberry Perl doesn't come with SSH modules pre-installed.

Here is how to insall

C:\>perl -MCPAN -e shell

cpan shell -- CPAN exploration and modules installation (v1.9304)
ReadLine support enabled

cpan> install Net::SSH
Fetching with LWP:
http://cpan.strawberryperl.com/authors/01mailrc.txt.gz
LWP failed with code[500] message[Can't connect to cpan.strawberryperl.com:80 (connect: timeout)]

As a last ressort we now switch to the external ftp command 'C:\WINDOWS\system32\ftp.EXE'
to get 'C:\strawberry\cpan\sources\authors\01mailrc.txt.gz.tmp3104'.

Doing so often leads to problems that are hard to diagnose.

If you're victim of such problems, please consider unsetting the ftp
config variable with

o conf ftp ""
o conf commit

I would like to connect to one of the following sites to get 'authors/01mailrc.txt.gz':

http://www.perl.org/CPAN/
ftp://ftp.perl.org/pub/CPAN/

Is it OK to try to connect to the Internet? [yes] yes
Fetching with LWP:
http://www.perl.org/CPAN/authors/01mailrc.txt.gz
LWP failed with code[500] message[Can't connect to www.perl.org:80 (connect: timeout)]
Fetching with LWP:
ftp://ftp.perl.org/pub/CPAN/authors/01mailrc.txt.gz
Fetching with LWP:
http://cpan.strawberryperl.com/modules/02packages.details.txt.gz
LWP failed with code[500] message[Can't connect to cpan.strawberryperl.com:80 (connect: timeout)]
Fetching with LWP:
http://www.cpan.org/modules/02packages.details.txt.gz
LWP failed with code[500] message[Can't connect to www.cpan.org:80 (connect: timeout)]
Fetching with LWP:
http://www.perl.org/CPAN/modules/02packages.details.txt.gz
LWP failed with code[500] message[Can't connect to www.perl.org:80 (connect: timeout)]
Fetching with LWP:
ftp://ftp.perl.org/pub/CPAN/modules/02packages.details.txt.gz
Fetching with LWP:
http://cpan.strawberryperl.com/modules/03modlist.data.gz
LWP failed with code[500] message[Can't connect to cpan.strawberryperl.com:80 (connect: timeout)]
Fetching with LWP:
http://www.cpan.org/modules/03modlist.data.gz
LWP failed with code[500] message[Can't connect to www.cpan.org:80 (connect: timeout)]
Fetching with LWP:
http://www.perl.org/CPAN/modules/03modlist.data.gz
LWP failed with code[500] message[Can't connect to www.perl.org:80 (connect: timeout)]
Fetching with LWP:
ftp://ftp.perl.org/pub/CPAN/modules/03modlist.data.gz
Creating database file ...

Gathering information from index files ...
Populating database tables ...
Done!
Running install for module 'Net::SSH'
Running make for I/IV/IVAN/Net-SSH-0.09.tar.gz
Fetching with LWP:
http://cpan.strawberryperl.com/authors/id/I/IV/IVAN/Net-SSH-0.09.tar.gz
LWP failed with code[500] message[Can't connect to cpan.strawberryperl.com:80 (connect: timeout)]
Fetching with LWP:
http://www.cpan.org/authors/id/I/IV/IVAN/Net-SSH-0.09.tar.gz
LWP failed with code[500] message[Can't connect to www.cpan.org:80 (connect: timeout)]
Fetching with LWP:
http://www.perl.org/CPAN/authors/id/I/IV/IVAN/Net-SSH-0.09.tar.gz
LWP failed with code[500] message[Can't connect to www.perl.org:80 (connect: timeout)]
Fetching with LWP:
ftp://ftp.perl.org/pub/CPAN/authors/id/I/IV/IVAN/Net-SSH-0.09.tar.gz
Fetching with LWP:
ftp://ftp.perl.org/pub/CPAN/authors/id/I/IV/IVAN/CHECKSUMS
Checksum for C:\strawberry\cpan\sources\authors\id\I\IV\IVAN\Net-SSH-0.09.tar.gz ok
Scanning cache C:\strawberry\cpan\build for sizes
DONE

CPAN.pm: Going to build I/IV/IVAN/Net-SSH-0.09.tar.gz

Checking if your kit is complete...
Looks good
Writing Makefile for Net::SSH
cp SSH.pm blib\lib\Net\SSH.pm
IVAN/Net-SSH-0.09.tar.gz
C:\strawberry\c\bin\dmake.EXE -- OK
Running make test
C:\strawberry\perl\bin\perl.exe "-Iblib\lib" "-Iblib\arch" test.pl
1..1
ok 1
IVAN/Net-SSH-0.09.tar.gz
C:\strawberry\c\bin\dmake.EXE test -- OK
Running make install
Prepending C:\strawberry\cpan\build\Net-SSH-0.09-cQfbZo/blib/arch C:\strawberry\cpan\build\Net-SSH-0
.09-cQfbZo/blib/lib to PERL5LIB for 'install'
Installing C:\strawberry\perl\site\lib\Net\SSH.pm
Appending installation info to C:\strawberry\perl\lib/perllocal.pod
IVAN/Net-SSH-0.09.tar.gz
C:\strawberry\c\bin\dmake.EXE install UNINST=1 -- OK

cpan>

ADSL on Ubuntu 9.10 (Karmic Koala)

Working with ADSL modems on Ubuntu 9.10:

Configure: pppoeconf

sudo pppoeconf

Check: rohit@lenovo:/etc/network$ more interfaces auto lo iface lo inet loopback iface dsl-provider inet ppp pre-up /sbin/ifconfig eth0 up # line maintained by pppoeconf provider dsl-provider auto eth0 iface eth0 inet manual
Start: pon
rohit@lenovo:/etc/network$ sudo pon dsl-provider Plugin rp-pppoe.so loaded. RP-PPPoE plugin version 3.8p compiled against pppd 2.4.5
Stop:poff
sudo poff
Log:plog
rohit@lenovo:/etc/network$ plog Dec 8 16:37:45 lenovo pppd[11102]: Plugin rp-pppoe.so loaded. Dec 8 16:37:45 lenovo pppd[11102]: RP-PPPoE plugin version 3.8p compiled against pppd 2.4.5

This method works well for BSNL's ADSL lines in Kolkata.

Download HD Trailers from Apple Trailers

Apple movies now 'stream' mov files, they cannot be saved (like they us. for example, if you view the HD movie trailer for the movie "GOOD" - the URL on the page leads to this stream:
http://movies.apple.com/movies/ifc_films/good/good_1080p.mov
Saving link creates a mov file of 75k - not the movie.

In order to download, add the letter 'h' (Hidden) to the movie file name:

wget http://movies.apple.com/movies/ifc_films/good/good_h1080p.mov

HD-Trailers.net has an article on this and a tip for Yahoo HD Movie Trailers - which I've never used...