Checksum App for Windows is a Worm!

Do not download the Checksum App from this site http://corz.org/windows/software/checksum/ . Once downloaded and run, a worm is loaded! Please see the image below. I'm glad I had my Lavasoft Ad-Aware running. Somehow my McAfee didn't detect it.

Internationalizing Numbers and Dates in Java

Locale

It's easy to be tempted to find a tool that can do a specific job. Consider formatting Dates and Numbers for different countries. If you do find a tool, great, but that's just making your application dependent on another tool and redundant since Java already provides you an API that does the job. There's a ton of language support that Java provides. If Klingon would become a language in the future, I'm sure Java will support it, hopefully here on earth.

A Locale in Java is a representation of the Language and the Country. It's also a class. Their's two properties of it that is a must - language id and country code each of which consists of 2 characters. Say if you have the German language, the locale is "de". If the language is still German but of the country Austria then the locale is "de_at". Locale for United States would be "en_US". There's a lot of websites that explains the different locales in different countries so I'm gonna leave it out.

When instantiating a Locale take its important to note what arguments are being passed to its constructor. I've come across a bug in one of the applications I've worked at where a line of code was passing a language id of this form languageId_countryCode where in fact it should only be languageId. If you need to pass both to the constructor, separate those two values out. See example below.

Locale germanLanguageId = new Locale("de_at"); // wrong assignment of locale value.

Locale germanLanguageId = new Locale("de"); // correcet assignment.

If both the language id and country code is needed then it should look like this.

Locale germanLanguageIdAndCountryCode = new Locale("de", "at");

Localized Or Internationalized Date

This is pretty straight forward. Just use the DateFormat class and pass in the right locale to get the right date display as shown below. When you actually want to get an instance of that class, use the factory method getDateInstance(int style, Locale locale).

Date date = new Date(); // Now the date can be of any type as long as its an object.

DateFormat dateFormat = DateFormat.getDateInstance(DateFormat.SHORT, locale);

String internationalizedDateString = dateFormat.format(date);

Now depending on your current Locale, let's say we are using the german language id de, their date would actually be in this format. This is due to the fact that we chose the format type SHORT.

dd.mm.yyyy // so for todays date in german it would be 14.11.2009, thank goodness I decided not to write

                   // this yesterday!

Now for locale en_US it would be 11/14/2009. Of course the code will take care of doing the formatting for whatever locale the application is supporting.

Localized or Internationalized Number

First thing to do is to always define the pattern of the number that we would like to show in different locales. When I say pattern, its how many zeros do we want to show and if we don't want to show anything is the value of the number is equal to zero, etc. The DecimalFormat class has a method applyPattern(String pattern) where we can plugin the pattern we want. We can get an instance of this class by using the factory method of NumberFormat and cast it to DecimalFormat as shown below.

float aNumber = 1,500.98;

DecimalFormat decimalFormat = (DecimalFormat)NumberFormat.getInstance(locale); //factory method

decimalFormat.applyPattern("#,##0.00"); //our cool pattern

String internationalizedAndFormattedNumber = decimalFormat.format(aNumber );

Now the pattern #,##0.00 means that the grouping of the number is in thousand. The hash sign is any digit, where if its zero it won't display that number. The zero in the pattern is a digit, where if its zero, it would display as 0. We are also restricting the number of digits after the decimal to two digits. Say if we have a number 1,450.50 in english locale (en_US), upon formatting this to german (de) it should show up as

1.450,50. The comma becomes as period and the period becomes a comma. Of course, if the number we are trying to format is in millions, then our pattern should look something like this #,###,##0.00.

If the pattern is missing, some numbers get displayed in some other locales and some would display just fine. If our number is 0 to be formatted in german it might appear like this 0.0E3 which just confuses our business people looking at our localized spreadsheet. A pattern must be supplied always to display the right number in the right format. If a number in our localized csv file displays as 12.61 and then opened in excel, that number will be translated into a date equal to December 1967. This is the side effect of not supplying a pattern.

One last tip, if an application is processing data and converting it into CSV for excel viewing, remember to replace all carriage line feeds or new lines embedded in a String value with an empty String (" "). Say we have the String "The value\n is supposed to be\n entered here". This value should be in one row on a cell right? But with the new line character \n excel will actually break those strings into 3 rows. It would be

                Cell A

Row 1     The value

Row 2     is supposed to be

Row 3     entered here

Have fun with I18N !

Working with Deleting/Updating a Set In Hibernate

If a hibernate mapping object contains contains a Set of objects, it's usually configured with the attribute 'set'. If you want hibernate to take care of updating/deleting those objects in the set from the database table, the set attribute cascade must have the value "all-delete-orphan" and properly handled in the code that tries to update it.

Say our parent object is Car and child objects are of type Part. Our set mapping would be of the form below

In order to make the changes to the set elements which are of Part type, the order of the update statements must be of this sequence.

Car car = manufacturer.getCar();

//we want to reduce the number of parts say from 10 to 8.

//If the two parts have the ids 9 and 10, then let's remove it

Set parts = car.getParts()

//iterate through the set and grab the objects with ids 9 and 10

Part part9 = (Part)iter.next();//assuming this is the 9th object in the set

Part part10 = (Part)iter.next();//assuming this is the 10th object in the set

parts.remove(part9);

parts.remove(part10);

//persist to database to update the records

saveOrUpdate(car); //this will merge the differences. the database record will //now have 8 rows instead of 10 for the parts. Also, you can use the merge(car) //method and it will work the same way.

//if you want to delete all parts since its no longer produced, just do this

parts.clear();

saveOrUpdate(car); //then there will be zero entries for the parts in the database table

//When it comes to adding a new part, just add the new part object and persist.

Part part11 = new Part();

//...fill properties

parts.add(part11);

saveOrUpdate(car);

JFreeChart LegendTitle Font Change

I've been trying to search the whole day yesterday on how to change the Font for a chart's legend title. JFreeChart doesn't have any good free reference documentation about this use case but I was finally able to figure it out by playing with LegendTitle's methods. The code below works for changing the Font of the current LegendTitle object.

LegendTitle legendTitle = chart.getLegend();

legendTitle.setItemFont(font);

legendTitle.setNotify(true);

The setNotify() method somehow alerts the different objects dependent on the LegentTitle and update it's state. If that is not called, nothing happens. If that method is set to true that means, the current change on the chart, specifically the LegentTitle, call all dependent objects and notify of this change and properly act on it.

iPhone Mobile Camp Atlanta

Last Saturday, July 18th, I went to attend the Mobile Camp Atlanta in Marietta the whole morning. There were alot of presentations to choose from crossing iPhone, Android, Palm-Pre, Windows Mobile, Blackberry development. But I mainly focus my interest in the iPhone seminars. The iPhone Development presentations covered best practices user interface design, Parsing XML, Core Data, JSON, Facebook and Analytics, Push Notification. I've attended all of them except for the last one.

I am not an iPhone developer but I'm looking into getting converted before the end of the year. After the end of the presentations, I'm decided to add in my skills Objective C Language which apparently iPhone is using. I believe phones in the not so distant future will become a critical part of the IT Enterprise Infrastructure in terms of delivering important, well formatted content to the business user or customer. I'm sure iPhone will become a great tool to realize this need.

Here's what I've taken for each presentations.

iPhone App Design Best Practices

1. Be true to the device. Know the capabilities of the hardware and what the API can provide.

2. Read the Human Interface Guidelines provided by Apple.

3. Start designing your app after item 2 & 3 are done.

4. Use DataObjects metaphor.

5. Start button design to 324x80 pixels and load it to a temporary button album in iPhone just to see how it fits.

6. Use a third party tool like iView for viewing prototypes of the interface design. You can also use iPhone Stencil created by Design Commission.

7. Remember that 3G S has warmer color that its predecessors.

8. Perform Paper Prototyping to understand the flow of the application.

9. Native apps are more comfortable with users than browser based ones. Native in this context is Objective C.

10. Do not hardcode the screensize, iPhone itself might change the lcd screensize in the future plus if you are developing on iPod as well.

11. Let the app be customizable by users to make it their own. User experience is a need.

12. Pixel density of iPhone is much higher than what you have in a laptop or desktop. The prototype from the laptop/desktop will look different when viewed from iPhone.

13. As a rule of thumb, always design apps both for iPhone and iPod.

Parsing XML using NSXMLParser

The NSXMLParser is SAX based parser. It rides on top of Cocoa. This goes hand-in-hand with NSDictionary and NSArray. NSDictionary allows you to store key/value pair. Like a Map in Java. To manage multiple occurence of same key, which are basically the xml nodes that you are trying to get onto, turn it into an array. This is only true for one level node. For deep level nodes, give the parser a hint then build the array.

Introduction to Core Data

Core Data in iPhone sense means Persistent Objects. All apps have a delegate. iPhone uses managed object model and context. There are 3 main APIs suitable to perform manipulating persistent objects - NSString, NSArray and NSDictionary.

JSON in iPhone

Use the JSON documentation from iPhone Noob. The only thing to do to deserialize JSON objects are to turn them into Dictionaries and Arrays. By the way, the debugging in XCode looks really cool.

FaceBook Connect in iPhone

I'm not really intested in developing applications via FaceBook. But if you do, all you have to do is download t FaceBook Connect Dev and apply to obtain "API Key" and "API Secret".

Analytics for iPhone

This was mainly a discussion about a product called Flurry, which is a better version of iTunes analytics. This comes close to Google Analytics. The presenter just basically showed us on the web his product he created and it does have much more meaningful data than iTunes does.

Integrating Author It Compiled Documents with Strutsonlin

Integrating the end product of a content management tool with a web application is painful. Even if the web application is built on top of an MVC framework. Author It is a CMS tool that will let you build an html based documentation. The focus on this discussion is integrating an end product which is an online help documentation with Struts. 

The documentation will allow users to view the help page regarding the current page by clicking on the Help button from the menu bar of the application. The user can also search for any specific help pages she/he wants to view. 

The integration effort requires several files to be created/edited as listed below

1. onlinehelp-config.xml

2. onlinehelp.jsp

3. common.js

4. mainMenuBar.jsp

5. web.xml

6. OnlineHelpProcessor class

and several classes as shown below from the diagram

onlinehelp-config.xml

This file contains the mapping between struts/tiles and the online help html documentation page through its numeric code which is the prefix to the file name of the help page (i.e. 207.htm). There are two parent nodes in it - nonupdate and update. 

nonupdate node contains mapping for help pages that are "only" non-editable web pages while the update node maps help pages to editable web pages.

The element is the tiles definition name attribute. This configuration is read and cached when the application starts up. Think of the element as the help page file name appended with .htm.

onlinehelp.jsp

This is the original index.htm file generated by Author It tool, the entry point for all help documentation pages. There are three modifications to this file - import of the taglib struts-bean.tld, bean definition of the onlineHelpCode and the SRC attribute value of the frame element. 

The import of the bean taglib is needed to extract the SessionData bean which lives in the session and get the onlineHelpCode string value from it and plop it to the SRC element. It is important that this file is located where the jsp files are.

common.js

Of course we want the help page to be in a new window. This is where we add a new function that will open a pop-up window when the help page is invoked.

mainMenuBar.jsp

This is where the Help button is located and a javascript function to invoke the online help documentation in a new window is identified. This jsp file contains all the buttons necessary to provide functionality for the user.

web.xml

We don't want the onlinehelp-config.xml file to be read each time we invoked the help pages. There's 3 ways to handle this file

1. Read this file, convert it to Java Bean/Map and put it in session.

2. Read this file, convert it to Java Bean/Map and put it in cache.

3. Read this file, put attributes in a Map and put it in a singleton object.

I prefer the latter for simplicity and less coupling though all of them promotes efficiency. If the bean is put in session we are introducing a tight coupling with the container's session implementation. If a third party caching mechanism is used then we are tied to that implementation.

In this example I used the third option and have a servlet intialize the class that reads this file which is called OnlineHelpProcessor (Singleton). The method called for initialization from this class is initializeOnlineHelpCodes(). Its just basically reading the file and putting attributes in a Map.

When the application starts up, the singleton class is initialized and all the elements from the config file is put into a Map ready for access anytime.

OnlineHelpProcessor

This class reads the onlinehelp-config.xml file and converts the elements into a key/value pair in a Map and provides a method that takes in a struts forward name argument and returns the onlinehelpcode value which is later used to load the right help page.

Using Chain of Responsibility Pattern

More than a year ago I was presented with a business requirement that has an accompanying process flow chart. The requirement  was basically validating an Insurance Agent. The following information were used to perform business rules/validation on a specific agent and see if he/she can do business at a certain area, or is able to use current authorization or if his/her current contract is valid. 

1. Authorization

2. Profile

3. Contract

4. Managing Agent

Looking from the document, there were sets of processes that contains business rules that needs to be performed but each process completion can lead to one or more possible processes to be run. This scenario is depicted below.

I chose a pattern to basically build the framework that will allow for easy integration of new processes into the mix and not complicate the validation implementation. If this is to be implemented without a framework, deeply nested if - else statements arises and increases cyclomatic complexity issues. The Chain of Responsibility Pattern seem to perfectly fit this requirement. This pattern comprise of several chains all link together and each chain is aware of if the request coming in is intended for it or not. If it is then it applies whatever process is needed on that request data. If not then it just passes the request to the next chain in the link. The process in the figure above is represented as a chain. In this case we have 9 chains in the link. 

Each of the 9 chains contains thick business logic validation implemetation that takes in an input and redirects the output to a different chain for further request validation. The process continues until one of the chains mark the validation process as completed and package the output data to whatever format the requester is expecting. 

This is the Class Diagram that reflects this design. In addition to the chain pattern, the Template Method Pattern was also used as can be seen from the hierarchy relationship of Chain interface, AbstractChain and its subclasses.

There are four things to consider from this diagram - Chain interface, ChainManager, AbstractChain and its implementation(chains).

Chain Interface

Defines the contract for implementing the methods add(Chain chain), processNext() and isLast(). The latter identifies a chain if its the last one to be executed. 

ChainManager

It is responsible for creating the chains, linking them altogether and kicking off the first chain. The pseudo code below shows this implementation.

Chain process1Chain = new Process1Chain();

Chain process2Chain = new Process2Chain();

Chain process3Chain = new Process3Chain();

Chain process4Chain = new Process4Chain();

Chain process5Chain = new Process5Chain();

Chain process6Chain = new Process6Chain();

Chain process7Chain = new Process7Chain();

Chain process8Chain = new Process8Chain();

Chain process9Chain = new Process9Chain();

DefaultChain defaultChain = new DefaultChain();

process1Chain.add(process2Chain);

process2Chain.add(process3Chain);

process3Chain.add(process4Chain);

process4Chain.add(process5Chain);

process5Chain.add(process6Chain);

process6Chain.add(process7Chain);

process7Chain.add(process8Chain);

process8Chain.add(process9Chain);

process9Chain.add(defaultChain);

// Assign the point of entry chain,

Chain entryPointChain = process1Chain;

//Kick of the first chain

entryPointChain.processNext();

AbstractChain

The process() method of this class reflects the Template Method Pattern. The power of polymorphism lies here. Its subclasses plugin the right logic at run time and these chains are decoupled from each other. 

As you notice the DefaultChain is included in the diagram. If you want to do some post-processing cleanup, provide the logic here.  

The beauty of this chain pattern is its extensible. From our example above, if we need to add another set of validation logic, all we have to do is create another chain and add it in the link and provide the business logic in its process() method. If a validation is no longer needed, just remove that chain from the link that implements the validation and the application should still run without any side effects.

When do we say its the right tecnology to use?

When is a technology the right tool to use? Do we influence the feature we want in an application from what is available from the technology or from what is required from the business requirement?

It's always a challenge to marry the right technology with the right requirement that would produce a fluid, solid application. One-Stop-Shop approach of resolving business issues is no longer becoming relevant in the global IT market. You pay so much that without realizing only a fraction of the features provided by that technology is only essential to what the business requires. A lot of companies now are resorting to only "get what you need" and "pay what you only need" mentality. This is where customization comes in. The beauty of customization is it delivers the solution that the problem it exactly expects. This strategy relieves companies from paying too much.

When the curse of overengineering starts to sink in, its a manifestation of wrong implementation. But what is overengineering? It's a combination of technology misuse and irrelevant complex implementation. If the implementation looks simple and it solves a business need then we just answered the first question above.

The danger comes when technology features molds how the application should behave. This deviates the application from concentrating into its prioritized functionalities and might deliver features not needed at all. Scope creep is the twin evil of this situation. Looking at it at the brighter side, this is the best opportunity to step back and review the requirements and correct and refine the implementation.

Implementing User Sessions for Crystal Reports Java SDK

If you are submitting requests for batch processing in Crystal Reports via its Java SDK, never login to the server per request. I've seen this implemented and it's just creating an unnecessary overhead. Here are the steps to create a session that would serve multiple requests.

1. Log on to Crystal Report Server

You call the metod CrystalEnterprise.getSessionMgr() which returns you an object of type ISessionMgr. From there call the method of the returned object below which returns an object of type IEnterpriseSession.

logon(String userName, String password, String ceServerName, String secEnterprise);

2. Create Logon Token

The IEnterpriseSession has a method getLogonTokenMgr() that returns ILogonTokenMgr. Call the method below to get the logon token string.

createWCAToken(String clientServerName, int validMinutes, int validNumLogon);

The validMinutes parameter represents how long do you want to maintain the session and the validNumLogon represents how many logons the client can perform in one session within the given valid minutes.

3. Logon to Crystal Server with Token

Call the method below from the class ISessionMgr and pass the logon token string that was previously generated. This returns to you a new IEnterpriseSession that has the token. All transactions will then use this session object until the minutes or log on counts are exhausted.

logonWithToken(logonTokenString);

Of course, somehow this session will expire later on and the client application need to reconnect. The good news is you just have to follow the same steps. On the other hand, if the session is still valid and somehow the application needs to reset the connection, always check for the session if it already expired. If it is, release the token and logoff with the following sequence then proceed to steps 1 through 3 again.

releaseToken(logonTokenString); // called from ILogonTokenMgr
ceSessionWithToken.logoff(); // called from IEnterpriseSession with token
ceSession.logoff(); // called from IEnterpriseSession without token

This approach will be very useful for batch processing. You want to keep the pipe full while the report server is processing. With on demand request report processing, you don't want to keep a session open for a long time.

Java Multithreaded Web Service Performance Tips

Right after Java 5.0 Concurrency Model was released, MultiThreaded Java Web Service application is always an interesting subject. It's easy to tweak codes that are slowing down the application. But it was always a challenge to tweak it at the container level.

A former colleague of mine and me worked to figure out why our Web Service Application was slowing down and always leaves lots of run-away processes. Here are some of the findings we had implemented to keep our application rolling! The sample codes below reflects a reporting process where requests to generate a report is made and how the threads handle the status of each requests.

Use Java Concurrency API

There are only three classes that I've used to implement a MultiThreaded WS Application.

Runnable, ScheduledExecutorService, and ConcurrentLinkedQueue. 

The ScheduledExecutorService implements ExecutorService and is of type Executor.  Executor is responsible for abstracting the gory details of implementing threads into simpler methods. And it works all the time.

ScheduledExecutorService allows you to provide what the initial time is to run the thread associated with it, at what frequency, the thread to run and the unit of time. Was very easy to use and you can configure the thread pool size as well. The application I wrote had 5 threads running concurrently and it never had any problems at all. They all shared the ConcurrentLinkedQueue object resource and I didn't use the synchronized keyword to make this object thread safe since by default it already is.

Using the Runnable interface is better than just extending the Thread class. It promotes less coupling. Here's a sample of how this thread is plugged-in to the ScheduledExecutorService.

Implement the ServiceLifeCycle

Implementing ServiceLifeCycle will allow you to put shutdown hooks and implement its destroy() method where you can manipulate how the spawned threads are to be shutdown/killed. The init(Object context) method must also be implemented.

Provide Shutdown Hooks

The shutDownMonitor() method is called and the ExecutorService's awaitTermination() method is invoked which basically gracefully shuts down the spawned thread in the WS Container according to the specified time (might be in seconds, minutes, hours). You can also force to shutdown all threads by calling the method shutdownNow() if it takes longer than the specified time.

Use "application" Scoping

When you use "request" scope as configured in your wsdd file, you are creating one instane of the service per request. If several requests were triggered, your application is also spawning threads per request. The previous request that was processed left a trail of run-away processes. If the scope was "application" it will create only one instance of the service that will serve multiple requests. Hence, when the application runs it does not create run away processes since we already have a shutdown hook to kill all of the spawned threads before the application completely shutdown.

I think it all depends on what an application is trying to achieve, it might have more complex business process than this sample code we have and it requires more classes to use from the concurrent api. But in a multithreaded web service application, you definitely want to use the application scope and implement a shutdown hook.

Jakarta Commons Net API

Jakarta Commons Net(JCN) Java Library is just one of those open source projects I've enjoyed working with. Specially its FTP API. Which is gonna be the focus of this discussion.

I've implemented a component that uses this library in a Web Service environment which basically connects to a remote FTP service then iterate through the list of directories, process the files and handle the reply codes properly. The list of files may go to several thousands. 

Here are the things I found really useful from JCN.

1. It automatically issues a new PORT command to the server so we don't have to worry about manually setting the port for different platforms (Unix, Linux, Windows, Mac, etc.) when connecting. Also validates data connections to client to ensure the request had originated from the intended host. We don't want our application deal with strangers eh?

2. It allows you to page through the list of FTP files. I think this is an awesome feature! From my previous project the list of files can span to several thousands. But through FTPListParseEngine class I was able to page through the files and process them in smaller chunks. This is a great performance boost. We only load the ftp file objects we currently need. Creating this objects are expensive. We keep the metabolism of an application running this way and saves us the evil of bottlenecks.  

3. Provides very easy to use and effecient way of handling FT Reply Codes. The codes are basically messages passed by the FTP Server indicating if the request was processed successfully or not. There are about 49 codes which you can check at this link http://www.turboftp.com/turboftp/manual/TURBOFTPFTP_Server_Reply_Codes.htm . After a command is issued to the server, the server returns an int value representing the code. Just capture this code and apply the behavior you want to happen in the application. 

This is a great way to isolate specific ftp issues and handle it properly so the application can still function, say even if the FTP service is down. Or if your previous request gave a code that identifies your request as not processed, you can resend it upto a certain number of times until you mark it failed. There's no point sending an ftp command or request if the data connection is not established, as shown below.

Also shown below is an example of flagging the reply if its ok or not with their matching simple behavior implementation.

I can see no reason why JCN can't be used. It provides the necessary security and performance gain we need. Plus it provides you an elegant way of handling FTP Reply Codes that you can use to manipulate the behavior of an application consuming an ftp service. Documentation is well written too.

AJUG (SOA and BPM Meeting) - My Take

Rick Geneva from Intalio company presented an interesting thought on how SOA and BPM work together tonight. So SOA is IT driven while BPM is business driven. Rick's title is a Process Expert. Alot from his slides pointed some technologies that instantiate an SOA architecture like WS, ESB, JMS..you name it. But I think we missed his concrete experiences with his customers/clients that he worked with to improve their business process. Though the framework "Process Driven Development" was mentioned there were no information as to how it is executed from inception to deployment. I agree with him that Process is very important. That's why we have these different SDLC methodologies like SCRUM, Waterfall, Extreme Programming, TDD, etc. But all of these methodologies are focused on building the software products. How do you tie this up with Process Driven Development (PDD)? So shall we say PDD is to Business Team and SDLC is to Development Team?

I'm interested to know how PDD works. I wished he talked more on that part. To me PDD sounds like a 6 Sigma methodology but the only difference is 6 Sigma not only focuses on the product but to the processes as well from business processes/operations down to the smallest detail of assembly of a product. Ten years ago I was a Process Engineer and got trained for Greenbelt 6 Sigma and was certified. A project was identified and was assigned to me and used 6 Sigma methodology to eliminate or at least minimize a product defect. The rest of the Process Engineers also had their own. After my 6 Sigma project was completed I saved the company $160,000 annualy and the product which I'm assigned had a better quality and the ROI is very tangible. I have not heard of any process driven development yet that had a very tangible ROI. The question is how do you measure a business process? With 6 Sigma you have 98.5% as your confidence level, computed statistically. I've seen alot of Software Companies using 6 Sigma as their methodology for delivering products. But I have not heard any welcoming news that it worked for them. I might be wrong. But I've never seen anything published.

Rick made it clear that there's no organization responsible for setting the standards for Process Driven Development and everything is still up in the air but alot of companies are already using software products that enable them to Choreograph and Orchestrate their services that will eventually define their BPM. I guess everybody will just have to wait and see. But there's definitely a need for a standard business process management.

BPMN was also mentioned which basically means Business Process Management Notation. It's another set of notations that BAs and Developers will have to learn. UML is there and I believe its sufficient to translate a descriptive business requirement into a more readable, simplified diagram. This is the part where I don't cast my vote in. I think the reason why management and development team cannot meet in the middle is because of unclear requirements sitting on top of a thick bureaucrat who is wrapped in his own fragile ego.

For now I consider BPM as a buzz. Its just basically a workflow. Nothing more. BPEL and the like on the other hand simplfies the complexity of implementing tons of workflows. But how do you measure its complexity to use such a product? Maybe if there are 100 steps in a workflow? Or if the integration points consumed in that workflow is more than 10 then do we consider that as a complex workflow?

Its hard to quantify the workflow metrics and I'm not sure how to do it at this point. We can't just dive in into our Infrastructure and start doing workflow metric testing to establish a benchmark. Also, you might have to hire a person with a position as Workflow Expert, if they do exists, to do the job.

In the end, all the business want is to see if they get any value from BPM. How much can they $ave, how much processes can they reduce and make efficient, and how can they make their IT Infrastructure flexible to accommodate the ever changing business needs.

Oracle BRM Portal Java PCM API Design Issues

I respect Oracle. They're awesome. They’re very smart and that’s why they bought Sun Microsystems. But no great companies are exempted from mistakes and great mishaps. I know you know some. I'm not surprised if you'd pick Microsoft. Dang Vista really hit them hard and that wasn’t there first. But I'm not gonna talk about Vista. I want to focus on what my experience was working with Oracle BRM Portal Java PCM API, specifically their FList class and Web Service Interface.

I have not used every single API but enough to point out what's wrong with it.

Here are the design issues I’ve found:

1. FList class is a Hashtable

2. FList data structure follows the anti-pattern “Yo-yo Problem”

3. Heavy use of Reflection

4. Vague documentation

FList class is a Hashtable

FList is the point of entry/gateway in the Oracle BRM Java API to initiate and create transactions. It can contain aggregates of different types of objects that represent the data and its relationship in the BRM database. The result coming back from any brm request is also a FList. So the brm request/response is of the FList form.

Now here comes the bad part. FList is a Hashtable! It’s thread-safe because it’s synchronized but at the same time greatly compromises performance. FList is very coarse-grained object that can have a deep tree structure. If you put it on a multi-threaded environment the application will bend down on its knees. So their Java PCM API is not scalable at all.

FList data structure follows the anti-pattern “Yo-yo Problem”

Think of this. All of the available object types in the Java PCM API are contained in the FList. You have to dig deep into it and know what type of object you are getting. The problem comes when you keep flipping from one object to another and that doesn’t clearly explain how these objects are related to each other and what subject do they convey. It’s confusing. Their PCM Java Documentation API is not enough to understand what are involved in a specific BRM process. You have to reference as well their Portal Documentation which is about 312MB containing around 10,756 files. If you need to understand something it’s like finding a needle in a haystack.

Heavy use of Reflection

I have never seen an application that uses heavy reflection till I came across FList. The image above shows how many times reflection was used to just grab one Sales Order through Oracle’s BRM Web Service interface. I’ve used JProfiler to take a snapshot of the call trace. The lookup of the class being invoked incurs the most overhead compared to its invocation (seehttp://www.jguru.com/faq/view.jsp?EID=246569). Oracle BRM WS has definitely some performance issues with its Java engine.

The Field class from the package com.portal.pcm made 12,855 lookups and off course another 12,855 invocations incurring 20 seconds to complete just these two tasks. Its ridiculous. What in the world is it doing???

Vague documentation

Oracle BRM is a beast. The documentation as well as mentioned above earlier. It’s so huge that you can easily get lost without the guidance of an Oracle Consultant. It’s not trivial. The goal of every complex system should be to provide enough and clear documentation. What really gets into my nerves is when we start customizing fields in BRM and the document doesn’t tell you what data you need to pass to FList to properly fetch what you want from BRM. Even outside of the customization discussion, you need several pieces of information from one page to another and combine them to formulate your desired FList structure.

This is just one of those tools where you definitely have to go to a training. I'd say hire one good Oracle Consultant and let your trained employees work with him to manipulate the behavior of their BRM tool. My current company is totally dependent on Oracle Consultants. We have 6 of them and they get paid a ton. I wish they invested in their employees and let them be trained and be guided by just one consultant.

Sun should treat eveything in Java as Objects!

I have posted a question in LinkedIn as shown below.

Do you think Sun will remove primitive data types(int, long, float, etc.) in their Java Platform?

I got 10 answers from different people and they are all against the idea of treat everything in Java as Objects. But I did clarify my point why I like the idea of Sun updating the Java Platform or creating new breed of Java Platform that will treat everything in Java as an Object. Here's my reason behind the question.

I have seen lots of Java code that keeps switching from a primitive type to a Wrapper class. Reflection API does not return the classe's primitive type as an object and of course you have to use Wrapper classes to later convert it to the right object type then switch back to a primitive type. Autoboxing just does the same thing. All of these are basically CPU overhead not heap overhead. RAMs are not that expensive and they get cheaper and cheaper while getting bigger in temporary storage. I guess I like the idea of having Java as everything is Object because that will minimize switching between primitive types and makes the code alot readable. But I do agree some of their points but I'm thinking not far from now it may happen. Heap size is easier to control than CPU time. There's a big difference between memory usage and cpu time in terms of overhead. The latter gives you a better sense of tuning performance. You would be surprised that scalability is actually affected more by CPU Time rather than Heap Size. If you have an out of memory issue, its almost always that you have run away processes that the CPU keeps it running till you run out of memory. There's something in the code that drove the CPU nuts.

I hope that explains my curiosity behind the question.

Singleton without "synchronized" keyword in Multi-Threaded App

When you ask a designer or developer to write a multi-threaded application, in their mind, the magic word - 'synchronized' immediately pops-up! That was my mentality as well until I came across this wonderful website explaining the different design patterns. It hit me hard. You can check it on this link http://www.oodesign.com/singleton-pattern.html.

Typical Singleton object looks like this:

//This is bad. The whole method is synchronized. If your method has
//thousands of lines of code then you are creating a bottleneck here... 

public class MySingletonObject{

private static MySingletonObject instance;

private MySingletonObject(){}

public synchronized static MySingletonObject  getInstance(){

if(instance==null){

instance = new MySingletonObject();

}

return instance;

}

}

or

//This is better. Only the block of instantiation is synchronized. But....
//There's always this null check overhead. Also, you still have the 
//synchronized keyword. It's still an overhead.

public class MySingletonObject{

private static MySingletonObject instance;

private MySingletonObject(){}

public static MySingletonObject  getInstance(){

synchronized(MySingletonObject .class){

if(instance==null){

instance = new MySingletonObject();

}

}

return instance;

}

}

//This is what you want. What? Why?...... Never underestimate
//the power of static modifier. Once you call this class
//say MySingletonObject.getInstance(), the classloader will automatically load
//and create the instance of MySingletonObject type.
//This member variable only has 1 instance all throughout the application
//since its at the class level. No more bottlenecks. No more waiting for the
//threads. They immediately get an instance. Looks very simple too...

public class MySingletonObject{

private static MySingletonObject instance = new MySingletonObject();

private MySingletonObject(){}

public static MySingletonObject  getInstance(){

return instance;

}

}

Using Rally Java Web Service API

 There are a number of developer tools that can be used from Rally ranging from integrating Rally to a different application(s) or just plainly extracting data from Rally. This link https://rally1-wiki.rallydev.com/display/Word/Developer+Tools provides necessary rally developer documentation.

Web Service API

Since Rally supports several implementation of their WS API this will mainly focus on SOAP in Java. You can find on this link https://rally1.rallydev.com/slm/doc/webservice/ the other implementations. The only thing I don't like about this API implementation is that you always have to pass the object reference through the wire to get the values of the object. The WS calls are so fine-grained that the number of objects queried is directly proportional to the number of round-trip calls. The sample usage of the SOAP in Java implementation is shown below in sequence.

Assuming our target of interest is to extract a Story from Rally. The Story in Rally is actually map to a SOAP object called HierarchicalRequirement. Always remember to use the read() method of RallyService^? object to grab the physical object from Rally.

1. Grab the WSDL from the current version of your Rally application which might have the form https://rallyx.rallydev.com/slm/webservice/x.xx/meta/34343483/rally.wsdl.
2. Generate the Java code from the given wsdl file. Their will be 3 packages generated - com.rallydev.webservice.domain and com.rallydev.webservice.service.
   • com.rallydev.webservice.domain contains all SOAP objects that represent the data in Rally.
   • com.rallydev.webservice.service contains the web service interface.
3.  Acquire connection from the web service endpoint and grab available Workspaces.

    URL url = new URL("https://rally1.rallydev.com/slm/webservice/1.10/RallyService");
   RallyService service = (new RallyServiceServiceLocator()).getRallyService(url);
    Stub stub = (Stub)service;
    stub.setUsername(rally_username);
    stub.setPassword(rally_password);
    stub.setMaintainSession(true);
   

   Subscription subscription = (Subscription)service.getCurrentSubscription();
    Workspace[] workspaces = subscription.getWorkspaces();
           if(workspaces==null || workspaces.length==0){
        errorBuf.append("The login credentials doesn't have any subscription or there are " + 
              "no Workspaces configured from Rally.");
       writeToFile(serviceBean, errorBuf.toString());
       return null;
    } 

4. If the target workspace is "IT: the next generation" then loop through the workspaces that matches that workspace.

   Workspace workspace = null;
     for(int i=0; i<workspaces.length;i++){
        WSObject wsObject = (WSObject)service.read(workspaces[i]);
         workspace = (Workspace)wsObject;
           String workspaceName = workspace.getName();
         if(workspaceName.equalsIgnoreCase("IT: the next generation" )){
             break;
         }
    }
    

5. Submit query and get results (DomainObject^?[]). The serviceBean.getQuery() is a name/value pair which might be of the form Release.Name= "Test Release For TWiki" AND ScheduleState^? = "Completed". Process each DomainObject^?.

   QueryResult queryResult = service.query(workspace, "HierarchicalRequirement", serviceBean.getQuery(), "", false, 1, 100);
     if(queryResult.getErrors().length>0){
         for(int i=0; i<queryResult.getErrors().length;i++){
             errorBuf.append("ERROR: ");
             errorBuf.append(queryResult.getErrors()[i]);
             errorBuf.append("\n");
         }
       writeToFile(serviceBean, errorBuf.toString());
       return null;
                }
   

   DomainObject[] domainObjects = queryResult.getResults();
    if(domainObjects!=null && domainObjects.length>0){
       List releaseNotesBeanList = new ArrayList();
       TwikiBean twikiBean = new TwikiBean();
       Map packageStoryMap = new HashMap(); 
   

      for(int i=0;i<domainObjects.length;i++){
          HierarchicalRequirement story = (HierarchicalRequirement)service.read(domainObjects[i]);
          Release release = (Release)service.read(story.getRelease());
          DateFormat dateFormat = DateFormat.getInstance();
          String releaseDate = dateFormat.format(release.getReleaseDate().getTime());
   

         if(story.getAttachments()==null || story.getAttachments().length==0){
             errorBuf.append(NO_ATTACHMENT).append("\n");
          }
          if(story.get_package()==null || story.get_package().equals("")){
             errorBuf.append(NO_PACKAGE_NAME).append("\n");
          }
          if(release==null || release.getName()==null 
                || release.getName().equals("")){
             errorBuf.append(NO_RELEASE_NAME).append("\n");
          }
   

         if(errorBuf.length()>0){
             errorBuf.insert(0, "ERROR: User Story ID: " + story.getFormattedID() + "\n");
             writeToFile(serviceBean, errorBuf.toString());
             continue;
          }
   

         ReleaseNotesBean releaseNotesBean = new ReleaseNotesBean();
          releaseNotesBean.setPackageName(story.get_package());
          releaseNotesBean.setUserStoryId(story.getFormattedID());
          releaseNotesBean.setUserStoryName(story.getName());
          releaseNotesBean.setReleaseDate(releaseDate);
          releaseNotesBean.setReleaseName(release.getName());
   

         Attachment[] attachments = story.getAttachments();
          Attachment attachment = attachments[0];
          attachment = (Attachment)service.read(attachment);
   

         AttachmentContent attachmentContent = (AttachmentContent)service.read(attachment.getContent());
          byte[] content = attachmentContent.getContent();
   

         String twikiTopic = new String(content);
          releaseNotesBean.setTwikiTopic(twikiTopic);
   

         releaseNotesBeanList.add(releaseNotesBean);
          prepareTopics(releaseNotesBean, packageStoryMap);
                       }
   

      twikiBean.setPackageStoryMap(packageStoryMap);
       twikiBean.setReleaseNotesBeanList(releaseNotesBeanList);
       return twikiBean;
    }