Setting the JVM character encoding on the AS400

When running a Java application (on any system) the JVM can be started with different character encodings. Typically it will pickup the system default based on the OS and locale system settings. For an excellent introduction to character encodings I can highly recommend Joel Spolsky’s article.

I recently encountered a problem with this on an AS400 where the Java application was trying to write a file out to the IFS file system. The filename in question contained Polish characters, or at least it was supposed to.

You can see the encoding in use by looking at the JVM properties against the job. Use WRKJOB to work with the job and take option 45 to Work with Java virtual machine. There are a couple of places you can then check.

  • Option 2 will show the system environment variables that were used to initialize the JVM
  • Option 7 will show the current Java system properties that are in use by the JVM

Searching online I saw many sources claiming that if you set the JVM argument file.encoding that this will set the default character encoding. On the AS400 this can be achieved by navigating to the SystemDefault.properties file and adding the line:

file.encoding=UTF8

Note: SystemDefault.properties can either reside in /QIBM/UserData/Java400 or in the user home directory for the user starting the JVM.

This may work when it comes to writing out the contents of a file but it has no effect on the encoding used to read or write filenames.

To properly influence the encoding used to initialize the JVM you have to set environment variables for the locale. In the Linux/Unix world this can be done with LC_ALL environment variable which can be set to something like en_US.UTF-8.

The AS400 isn’t a *nix platform so LC_ALL does not apply and the JVM is an IBM platform specific implementation. By looking at the environment variables against option 45 in WRKJOB and trial and error I managed to find that setting the following two environment variables did the trick.

Using ADDENVVAR add the following two environment variables:

QIBM_PASE_CCSID = 1208
PASE_LANG = EN_US

You can try different combinations of CCSID and locale depending on your desired character encoding. I set this up so we would be using UTF-8.

A list of locales can be found here. Note that en_US is either ISO8859-1 or 8859-15 whereas EN_US is UTF-8.

 

Advertisements

Calling RPG on the AS400 from Java

RPG is the native language on the IBM as400 midrange server (aka iSeries, system i and now just “i”). In a recent project I had to find a way to call a number of RPG programs from a Java application. If you’re in this situation then there are a few options available.

  • PCML (the subject of this article)
  • SQL Stored procedure
  • Integrated web services server (IWS)

Stored procedure
One possibility is to write a SQL stored procedure using RPG which could be called from Java using JDBC. This option may not be viable depending on the parameters your program needs. A stored procedure is good for returning result sets of records but you can only return one and you can’t pass one in.

Integrated web services server (IWS)
If you want to quickly expose an RPG program as a web service then you might want to look at IWS. This is a quick way to get up and running but there are a number of limitations.

  • If you want multiple operations on the same wsdl you have to write them all as procedures in the same service program
  • If using a service program IWS only supports up to 7 parameters including both inputs and outputs
  • IWS only supports contract last development. In other words you have to write the code first to get the wsdl.
  • The generated wsdl has a number of duplicated elements which you have to manually remove to tidy the appearance.
  • If you change the parameters or operations you have to go through the whole wizard again on each machine you deploy to.
  • Arrays are fixed size in RPG so IWS always returns all elements, even if some are simply blanks.

PCML
The most flexible method is Program Call Markup Language (PCML). This is an API that IBM provided for just this scenario. PCML is an XML language for defining the parameter list for an RPG program. This can then be used from a Java application.

Generating the PCML
You could write the pcml file by hand but a better way is to get the RPG compiler to generate it for you. First lets write a simple RPG program that we want to call.

D CONVTEMP        PR                  Extpgm('CONVTEMP') 
D  iCelsius                      9  3 Const                   
D  oFahrenheit                   9  3                    
D CONVTEMP        PI                                     
D  iCelsius                      9  3 Const                   
D  oFahrenheit                   9  3                    
 /free                                                   
   // We could get an API to return whatever we like here
   oFahrenheit = ((iCelsius * 9) / 5) + 32;              
   Return;                                               
 /end-free                                               

Yes it’s the cliche web service example to convert temperature from Celsius into Fahrenheit. It’s a good one to start with though because it is has both an input and an output but is still fairly simple. Note that the input parameter has been defined as a const, this is significant when we generate the PCML.

To generate the PCML from here you need to prompt compile and set the following options. Set PGMINFO to *PCML and INFOSTMF to a path on the ifs where you want your generated file to go e.g. /mylib/CONVTEMP.pcml. Doing so gives the following PCML.

<pcml version="4.0">
   <program name="CONVTEMP" path="/QSYS.LIB/PCMLTEST.LIB/CONVTEMP.PGM">
      <data name="ICELSIUS" type="packed" length="9" precision="3" usage="input" />
      <data name="OFAHRENHEIT" type="packed" length="9" precision="3" usage="inputoutput" />
   </program>
</pcml>                                              

We now have an XML file that describes how to call this program. Notice that the iCelsius parameter has been set to input but oFahrenheit is inputoutput. This is a result of setting iCelsius to a const parameter. When making a PCML call you must set a value for all input parameters. The default is inputoutput which can go both ways but is inconvenient if you don’t have an input value to set. Unfortunately there’s no language feature in RPG to set a parameter to output only so you have to adjust these manually.

      <data name="OFAHRENHEIT" type="packed" length="9" precision="3" usage="output" />                                            

Java dependencies
To make a PCML program call you just need the jt400 jar on your classpath. You can either use the IBM version that comes bundled with the AS400 or the open source JTOpen version.

If you use maven then you can simply declare it as a dependency like this.

<dependency>
   	<groupId>net.sf.jt400</groupId>
	<artifactId>jt400</artifactId>
	<version>6.7</version>
</dependency>                                       

This works fine but sadly the JTOpen developers stopped publishing to maven central at version 6.7 (current version is 7.10 at time of writing).

Calling the program
This is everything you need to make the program call. The Java class below is a simple test that opens a connection, calls the program and returns the result.

import java.math.BigDecimal;

import com.ibm.as400.access.AS400;
import com.ibm.as400.data.PcmlException;
import com.ibm.as400.data.ProgramCallDocument;

public class ConvertTemperature
{

	private AS400	as400;

	public ConvertTemperature()
	{
		as400 = new AS400("SYSTEM", "USERNAME", "PASSWORD");
	}

	public BigDecimal celsiusToFahrenheit(BigDecimal celsius)
	{
		BigDecimal fahrenheit = null;

		try
		{
			ProgramCallDocument pcml = new ProgramCallDocument(as400, "CONVTEMP");

			pcml.setValue("CONVTEMP.ICELSIUS", celsius);
			boolean rc = pcml.callProgram("CONVTEMP");
			if(rc)
			{
				fahrenheit = (BigDecimal) pcml.getValue("CONVTEMP.OFAHRENHEIT");
			}
		}
		catch(PcmlException e)
		{
			e.printStackTrace();
		}

		return fahrenheit;
	}

	public static void main(String[] args)
	{
		ConvertTemperature ct = new ConvertTemperature();
		ct.celsiusToFahrenheit(new BigDecimal(25.2));
	}

}

The second parameter to the ProgramCallDocument constructor is the path on the classpath to the PCML xml document. I created a file called CONVTEMP.pcml and put it in the src/main/resources folder. To keep things simple this is the root classpath folder, the .pcml suffix is not required as it is implied.

The PCML API will automatically handle converting Java types to AS400 types and back again. In this example the packed decimal from the AS400 becomes a BigDecimal in Java.

This is obviously a basic example that works as a proof of concept but there are a few additions worth mentioning if you want to use this in a production environment.

Adding connection pooling
Each time you create an AS400 object you’re opening a physical connection to the AS400. Each new connection creates a new job on the AS400. It’s obviously a bit wasteful to then throw this away and start with a fresh connection on the next call. A much better solution is to create a connection pool.

First you need to create the connection pool object. This code should live in it’s own class so the pool can be shared by different parts of the application. You could also load a properties file from the classpath to set the connection pool properties.

AS400ConnectionPool pool = new AS400ConnectionPool();

Now each time you want a connection you simply ask the pool. If no connections exist then one will be created.

AS400 as400 = pool.getConnection("SYSTEM", "USERNAME", "PASSWORD");

Remember to always return the connection back to the pool once you’re finished with it. This should be done in the finally section of the try/catch block to ensure the connection is returned if an exception is thrown.

pool.returnConnectionToPool(as400);

Setting a library list
The PCML file generated had a fixed path to a specific library. In practice you may find the program exists in different libraries and you want to use the one at the top of the library list. To do this we must first change the PCML file to not hardcode the library.

Change this:

<program name="CONVTEMP" path="/QSYS.LIB/PCMLTEST.LIB/CONVTEMP.PGM">

To this:

<program name="CONVTEMP" path="/QSYS.LIB/%LIBL%.LIB/CONVTEMP.PGM">

Next we need an event listener that will set the library list when a new connection is opened.

import com.ibm.as400.access.AS400;
import com.ibm.as400.access.AS400Message;
import com.ibm.as400.access.CommandCall;
import com.ibm.as400.access.ConnectionListener;
import com.ibm.as400.access.ConnectionPoolEvent;
import com.ibm.as400.access.ConnectionPoolListener;

public class AS400ConnectionPoolListener implements ConnectionPoolListener
{
    @Override
    public void connectionCreated(ConnectionPoolEvent event)
	{
		AS400 as400 = (AS400) event.getSource();
		CommandCall command = new CommandCall(as400);
		try
		{
            String liblCommand = "CHGLIBL(QTEMP PCMLTEST QGPL)";
			if(command.run(liblCommand) != true)
			{
				// Show the messages (returned whether or not there was an
				// error.)
				AS400Message[] messagelist = command.getMessageList();
				for(int count = 0; count < messagelist.length; count++)
				{
					// Show each message.
					System.out.println("System message: " + messagelist[count].getText());
				}
			}
        }

	@Override
	public void connectionExpired(ConnectionPoolEvent event)
	{
		// Not currently overriden
	}

	@Override
	public void connectionPoolClosed(ConnectionPoolEvent event)
	{
		// Not currently overriden
	}

	@Override
	public void connectionReleased(ConnectionPoolEvent event)
	{
		// Not currently overriden
	}

	@Override
	public void connectionReturned(ConnectionPoolEvent event)
	{
		// Not currently overriden
	}

	@Override
	public void maintenanceThreadRun(ConnectionPoolEvent event)
	{
		// Not currently overriden
	}
}

Finally the event listener needs to be registered as an observer of the connection pool.

pool.addConnectionPoolListener(new AS400ConnectionPoolListener());

Summary
This is a basic example that shows how to call an RPG program from Java. To brush this up a bit for production you only really need a few classes to wrap the connection pool and loading of properties. This would allow you to set the library list on different servers with a simple properties file. I would use spring to load the properties and register a bean that holds the connection pool. If you have to support multiple environments then you could set the library list each time you get a connection. Alternatively it might be more efficient to pass the environment to the RPG program and handle it on the AS400.