Java Based Internationalization Made Easier

Internationalizing Java applications includes highly repetitive tasks.  The standard method is to use a property file per supported language, which effectively contains a look-up table of content that is presented in the user interface.  Hard-coded content displayed in the user interface is moved into properties files, and is replaced with unique keys.  Initially, the following syntax was used for naming keys:

• <source file name>.<description>

The format allowed developers to quickly determine where the key is used in the source code.  It also helped provide translators the general area of the user interface the content is used, thus providing a bit of context.  Later in the project, translators requested further information to narrow down context even further.  The following format, which includes a look-up value used to indicate descriptor (button, label, link, title, etc...), was used:

• <source file name>.<descriptor>.<description>

Note that keys are only used once within each application to maximize translation flexibility.  We wanted to avoid forcing translators to use the same content in multiple places, which could result in incorrect contexts.
In general, internationalizing applications is not a technically difficult task.  However, it is time consuming, repetitive, monotonous work and therefore prone to errors.  Anything that can be done to minimize errors due to this is highly recommended.
Syntax highlighting source code editors are hugely advantageous when internationalizing applications.  These editors allow developers to efficiently scan source code for literals, which is where content that requires translating resides.
However, these editors can't provide syntax highlighting for languages within languages which is sometimes the case.  For example, Java source code literals can contain HTML source, which in turn contains JavaScript.  Source inside of source is important because different escape sequences are required.

To simplify the effort, a class should be used to maximize code reuse in the application.  Create methods in the class that perform common tasks such as returning text based on a properties file key, adding escape sequences to content based on the target language (HTML, JavaScript, JavaScript within HTML) and formatting numbers.

I found that using tools like AutoHotKey to automate common tasks was extremely beneficial.  I created a shortcut that copied the currently selected text to the clipboard and replaced the text with a method call to the internationalization object, including the source file name as the argument.  The appropriate quotes and concatenation operators were also included.  Then I only had to type in a description for the text, and put the corresponding entry in the appropriate property files.  With minimal ingenuity, AutoHotKey can automate most of the task of inserting the entry in the property files as well.  Basically any task/key sequence you are doing repeatedly can be added to AutoHotKey. 
After changing my development to use AutoHotKey, I found that the number of errors was significantly reduced.  Originally, I had a relatively large amount of typos, missing and out of order letters, etc… in the source code and property files.  Afterwards, these were almost entirely eliminated.  Furthermore, I was internationalizing the source code much faster than before. 

Tips to Maximize Productivity - AutoHotKey and KeyText

Problem:

There isn’t enough time in the day to get everything done.  There’s always more to do, and the general pace of things seems to be ever increasing.  Optimizing time usage is an ongoing challenge, as well as trying to avoid or minimize mundane and repetitive tasks.

Solution:

Using tools to improve productivity and carry out repetitive and mundane tasks frees you up for more important tasks.  They reduce errors that people commonly make when performing tasks over and over.  Last, they can quite often perform tasks faster.

AutoHotKey is an open source Windows utility provided under the GNU General Public License.  KeyText is a Windows utility that is free to evaluate for 30 days. 

AutoHotKey and KeyText are extremely useful to automate repetitive and/or common day-to-day tasks such as typing class names, method names and property keys along with the appropriate syntax for ending and restarting literals within the source code.  They can also be used to search for content, copying it to the clipboard and replacing it, as well as pasting clipboard contents into properties files.  Using these tricks can significantly reduce the amount of typos and errors.  Furthermore, less typing is required, which reduces development time and effort.

AutoHotKey and KeyText can also be used to automate interacting with relatively complex user interfaces.  Although the development interface is limited, the utilities can produce mouse clicks (based on x-y coordinates relative to the selected window) and keystrokes.  When combined to form complex sequences, it is possible to automate compiling, releasing, creating backups, logging into applications and other repetitive time consuming tasks.  These sequences are mapped to keyboard shortcuts, allowing developers to program the utilities with multiple different sequences, each being used to simplify a specific task.

As all developers know, just because an application compiles doesn't mean that it works.  For example, a common problem when internationalizing applications is missing keys in the property file.  To combat this, try-catch statements can be added to the appropriate methods, returning content an easily noticeable error statement that includes the name of the missing key.  The message can also be written to a log file or the console, further easing development and finding bugs.

Here’s a list of some of the tasks that I have recently used these utilities for:

• Login to application being developed
• Automatically navigate to part of application being developed
• Internationalization of applications 
• Searching for content to move to properties file
• Copying, replacing (with keys using a standardized format) and then pasting the content
• Generating load on a web server by triggering numerous requests (50+)

 

Coupling , Cohesion, and the Importance of Escaping Delimiters

Problem:

A customer was unable to debug or reproduce why a web-based application was crashing periodically.  The crashes were becoming more frequent, and increasingly affecting their business and customer satisfaction.

Steps used to solve the problem:

1.       Rule out performance issues through performance testing

2.       Aggregate and Review logs for finding issue trends

3.       Improve error tracking in the application

4.       Wait till issue reproduced

5.       Used additional logging and new tools to discover underlying issue

6.       Develop fix

7.       Confirm resolution

Tools used to solve the problem:

1.       Rational Performance Tester

2.       Sikuli

Cause of the problem:

Search entry string with a tab.

Tips to avoid in the future:

1.       Properly escape characters used to delimit data/fields

2.       Never make assumptions when dealing with user inputted content

3.       Use design patterns including encapsulation and low coupling

4.       Use debug statements in logging

Brief Background

I have been working on a project for a client on and off for approximately a month.  I was engaged to debug and solve an intermittent problem in a Java client-server application that is over ten years old.  Periodically, the server application would get locked up, using 100% of the CPU, until it was manually killed and restarted.  During this time, the client applet was unbearably slow or completely didn’t work.  The problem had been around for a long time, but was starting to occur more frequently, likely due to more customers using the application.

Analysis

After doing load testing using Rational Performance Tester (RPT) and monitoring the memory usage of the server application, it was determined that the issue was not caused by loading.  I successfully verified that the application ran fine with approximately 10x the maximum number of actual users observed in production.

While performing the load testing, I analyzed server log files.  They had been looked at previously, but nothing in particular stood out.  I looked for various patterns around the times where problems were logged by operations personnel.  Luckily, the developers created log entries when commands were started and completed.  After matching up intertwined start and complete logs for commands using Cygwin scripts and Excel, I discovered that a specific command was started but never completed right around the time when the issue occurred.  This was found in all of the dozen log files analyzed.

After narrowing the cause, I focused my testing using Sikuli to exhaustively test the functionality in question.  I also added more logging to better understand how to solve the problem.  I was not able to reproduce the problem, so we decided to go live with the version of the application with added logging and wait for it to occur again.

The problem occurred soon after and I looked at the generated log files.  Immediately, I was able to determine the cause and recreate the problem.  In the application, messages are being sent from the client applet to the server using delimited strings.  However, the delimiters were not being escaped.  Therefore, whenever a user entered a delimiter in the user interface, the corresponding message was not formed properly, causing the server application to improperly process it.

In the case of this application, message contents are used as arguments in SQL procedures with minimal error checking.  Invalid and/or missing arguments were causing infinite results.  So the server application was becoming locked up attempting to fetch and process the results.

There are several modifications I made to the client-server application to solve the issue and make it generally more robust.

First, I modified and tested changes to the server application to ensure the changes were effective.  I modified the loop that reads from the database to include a loop counter, and to exit after an overly large number of iterations.  This change eliminates the infinite loop, eliminating the need for operations personnel to restart the server application.  The break condition was made overly large to guarantee it would not exit prematurely.

The algorithm used to process the message was also improved.  It was modified to handle, as best it reasonably can, extra delimiters in the command.  It also now validates arguments passed to SQL procedures.

Second, I modified and tested changes to the client applet.  The class that packages messages was modified so that delimiters are removed from the fields put in the message.  The modifications were complicated by the fact that in some cases the class was not used to create the message, rather the message was created manually with string concatenations and hard-coded delimiters (poor encapsulation).  It was extremely time consuming to go through all of the messages, verifying they are created properly, modifying them so that they are using the class for packaging, and to test and verify all the changes.  The lack of encapsulation and high coupling made the client-server application far more difficult to maintain.

Conclusion/Tips for Developers

This engagement highlights the importance of encapsulation and low coupling.  If the class designed to package messages was consistently used, reducing coupling and making it well encapsulated, eliminating the delimiters in the fields of the message would have been much simpler to add.  A small change to the class would have sufficed.  Instead, the class had to be modified, code had to be changed to use the class for creating the messages, and a bunch of code had to be traced so that it was better understood.

Last, escaping delimiters is extremely important and no assumptions should be made when dealing with user entered content.  Escaping delimiters is an extremely common in programming.  Assuming a class will only be used for a specific purpose, for example content without tabs, can lead to future issues headaches when the class is reused.  In this case, the tab character was used as a delimiter.  Users should never enter the tab character, and in fact it is difficult to do so because pressing a tab changes the focus to the next element.  However, it is still possible to paste tab characters into the user interface, which is what users were doing.