Paul René Jørgensen

I’ll post this piece of code since it could come in handy for more developers than me.

	private static HashMap decode(int[] data) {
		HashMap m = new HashMap();

		// Extract date information
		int date = data[0] << 8 | data[1];
		int year = 2000 + (date & 0x7f); // 7 bits
		int month = (date >>> 7) & 0xf; // 4 bits
		int day = (date >>> 11) & 0×1f; // 5 bits

		// Extract time information
		int millis = data[4] << 8 | data[5]; // 16 bits
		int time = data[2] << 8 | data[3];
		int minute = time & 0x3f; // 6 bits
		int hour = (time >>> 6) & 0×1f; // 5 bits
		int second = millis / 1000;
		millis = millis % 1000;

		// Construct Date object
		Calendar c = Calendar.getInstance(TimeZone.getTimeZone(”GMT+00″));
		c.set(year, month-1, day, hour, minute, second);
		c.set(Calendar.MILLISECOND, millis);
		m.put(”gpsTime”, c.getTime());

		// Extract validity
		m.put(”valid”, (time >> 31) == 1);

		// Extract latitude
		int latValue = data[6] << 24 | data [7] << 16 | data [8] << 8 | data[9];
		if((latValue>>>31) == 1) latValue = - ((~latValue) + 1);
		m.put(”latitude”, latValue * 0.0000001);

		// Extract longitude
		int lngValue = data[10] << 24 | data [11] << 16 | data [12] << 8 | data[13];
		if((lngValue>>>31) == 1) lngValue = - ((~lngValue) + 1);
		m.put(”longitude”, lngValue * 0.0000001);

		// Extract speed
		int speedValue = data[14] << 8 | data[15];
		m.put("speed", speedValue * 0.01);

		// Extract course
		int courseValue = data[16] << 8 | data[17];
		m.put("course", courseValue * 0.01);

		// Extract altitude
		int altValue = data[18] << 8 | data[19];
		m.put("altitude", altValue);

		return m;
	}

I hereby put this code into the public domain.

I just stumbled upon an article by Werner Vogels about consistency in distributed systems. It was an eye opener and I recommend everyone interested to read it.

Eventually Consistent: “This is a specific form of weak consistency; the storage system guarantees that if no new updates are made to the object, eventually all accesses will return the last updated value. If no failures occur, the maximum size of the inconsistency window can be determined based on factors such as communication delays, the load on the system, and the number of replicas involved in the replication scheme. The most popular system that implements eventual consistency is DNS (Domain Name System). Updates to a name are distributed according to a configured pattern and in combination with time-controlled caches; eventually, all clients will see the update.”

In the same reading frenzy I also read these related pages:

• http://www.readwriteweb.com/archives/is_the_relational_database_doomed.php
• http://belaban.blogspot.com/2009/01/replcache-storing-your-data-in-cloud.html
• http://commons.apache.org/sandbox/pipeline/
• http://www.jgroups.org/demos.html
• http://www.helidb.org/

My employer Adresseavisen (local newspaper in Norway) released a Google Maps based property transfer database I’ve written for them today.

You can visit the application here: Eiendomsdatabasen

• Usemon presentation is available from Slideshare
• Usemon Live demo video is available here

Tonight I upgraded one of my Sun Spots to the new beta release of the blue firmware. One of the changes were an important bugfix in the UART code of the daughter board handling the UART buffers.

Anyway, after upgrading I wrote some test code to display the NMEA data from the GPS and guess what.. It worked!

        EDemoBoard edb = EDemoBoard.getInstance();
        edb.initUART(edb.SERIAL_SPEED_4800, edb.SERIAL_DATABITS_8, edb.SERIAL_PARITY_NONE, edb.SERIAL_STOPBITS_1);
        while(true) {
            try {
                System.out.print((char) edb.readUART(1000));
            } catch (IOException e) {
                e.printStackTrace();
            }
        }

And the output (No fix. I was in my livingroom)..

Squawk VM Starting (blue-080718)...
$GPGGA,220058.000,,,,,0,00,50.0,,M,0.0,M,,0000*40
$GPGSA,A,1,,,,,,,,,,,,,50.0,50.0,50.0*05
$GPRMC,220058.000,V,,,,,,,080808,,*2A
$GPGGA,220059.000,,,,,0,00,50.0,,M,0.0,M,,0000*41
$GPGSA,A,1,,,,,,,,,,,,,50.0,50.0,50.0*05
$GPRMC,220059.000,V,,,,,,,080808,,*2B
$GPGGA,220100.000,,,,,0,00,50.0,,M,0.0,M,,0000*4C
$GPGSA,A,1,,,,,,,,,,,,,50.0,50.0,50.0*05
$GPGSV,3,1,12,03,67,195,,06,61,164,21,19,57,270,,18,48,083,17*71
$GPGSV,3,2,12,22,46,145,17,21,26,080,,08,20,318,32,15,20,030,*74
$GPGSV,3,3,12,26,17,013,,27,12,293,,07,11,282,,16,10,185,*71
$GPRMC,220100.000,V,,,,,,,080808,,*26

Sorry for not posting any progress on this for some days, but I’ve started in my new day job and have been pretty tired in the evenings. Hopefully this will change soon, but during some spare time tonight I was able to create the proper cable to be able to interface the GPS with the Sun Spot.

I just wanted you to see that tiny red light on the GPS indicating that it’s properly powered and connected to the Sun Spot. Now that I’ve prototyped all the hardware I’m ready to tackle the the real challenge, to code this bugger! That will be a lot of fun!

Just a reminder at the end. I hope to meet a lot of you guys at JavaZone, 17.-18. September in Oslo. I’ll be at the speakers dinner on the evening the 16, but hopefully we’ll get some time to grab a few beers and talk about old times during the conference..

I just wanted to let you know about todays progress. The GPS module landed in the mailbox this morning, but I forced myself not to rush and wire it up to the Spot right away. I know that this GPS module is compatible with the Spot so there was no reason to rush it.

What I was more concerned with was the enable/disable switch and the rudder pass through signal. I started out wiring up the RC receiver to the spot and wrote some lines of code to get the pulse length of a signal connected to one of the Dx pins of the Spot.

    private int getRCPulse() {
        EDemoBoard edb = EDemoBoard.getInstance();
        IInputPin pin = edb.getIOPins()[EDemoBoard.D3];
        return edb.getPulse(pin, true, 30);
    }

These few lines of code should give me the length of the pulse coming in on the D3 pin every 30 ms measured in microseconds. And guess what happened when I connected the battery? It worked!!

Next up was to prototype the rudder servo pass through. I added some more wires to the Spot connecting the servo signal cable to one of the Hx pins and made sure the servo was powered by the RC receiver.

        Servo s = new Servo(EDemoBoard.getInstance().getOutputPins()[EDemoBoard.H1]);
        while(true) {
            int l = getRCPulse();
            s.setValue(l);
            System.out.println(l);
            Utils.sleep(1);
        }

This code reads the rudder RC channel and passes the value on to the connected servo. Amazing as it sounds, but it actually works!

I now have a spot decoding the RC receiver signal and later outputting it to a connected servo, passing it through the spot as if it was not there! This is what is supposed to happen when the autopilot is disabled. When I hit the gear switch on my transmitter the autopilot logics should kick in. But that is another story..

Crazy me have been thinking about this for some time now. Those who know me know that I’m trying to make a part time business taking photographs from the air using RC model aircrafts. I have a dedicated page for some of the videos I’ve been making at http://paulrene.no/index.php/aerial-photography/. I even set up a page for my business at http://settfraluften.no (Norwegian only).

Anyway; During spring this year I got hold of a Sun Spot development kit and I’ve been playing around with it for a while. When writing a program to control servos using the built in accelerometer the idea struck me that this would be a nice platform for an autopilot. I’ve never worked in this field before so this is all new to me and a bit of a challenge. But if I can pull this of it would be a nice project and maybe something that would inspire others to do the same and improve it.

One of the first things I did was to assemble a list of parts:

• Sun Spot - http://www.sunspotworld.com
• Pressure sensor - MPX5100AP pressure sensor 15-115kPa
• GPS module - GPS-module EM-406a
• Headers - Stiftlist 2×10poler rak Au
• Gyro - MKS 380 Aeroplane Gyro

Along with this I need servos, RC receiver, wires and connectors.

The feature list as I see it just now will include:

• Way point navigation using rudder control
• Altitude hold
• Autopilot enable/disable using separate RC channel (landing gear channel)

I’ll continue to write about this as the project evolves. Stay tuned, this will be fun!

Did you ever want to be the Big Brother of the Java Virtual Machine? Being able to do non intrusive monitoring of method calls without access to the source code, performing this in a production environment without a noticeable performance hit?

Do you want to know who is doing what to whom, that is, who invokes what method in context of which principal? Making certain that memory consumption is dynamically adjusted to avoid out of memory situations? Would you also like to use it in your legacy applications running pre Java 5 as well as utilize the new Java agent interface of the newer JVMs?

You know you have to collect vast amount of data and process it effectively and in real time. Finally you figure out that the amount of data exceeds the capabilities of a traditional data model and you decide that the only solution is a dimensional data model, using MDX for multi dimensional analysis.

There are numerous tools available for profiling and measuring your Java programs. However these tools are either: a) aimed at profiling or measuring your code in a development environment; b) expensive; c) not open source; d) requires a Ph.D. in debugging and profiling e) any combination of the above

Usemon attempts to alleviate this by providing an open source tool which is easy to use, collects data from environments based upon Java 1.4 through 6, has a flexible dimensional data model stored in a SQL database and attempts to be non-intrusive.

Usemon hooks into the Java virtual machine through a modified class loader on older JVMs or via the Java Agent Interface from version 5 and up. It automatically monitors Java EE components (EJBs, Servlets, MDBs, Queues, SQL Statements, etc), but can be configured to inspect any class in your JVM. Data is multicasted from agents to one or more collectors and stored in the DBMS of your choice. Usemon comes with a complete configuration for MySQL.

The overhead of running Usemon is negligible and thus you may collect run-time statistics around the clock all year, which is rather fascinating, given that you have enough disk space.

Usemon was developed for large Java EE environments at Telenor, which is one of the worlds largest telco to fulfill these requirements. You may utilize it in any environment in which you have control over the Java Virtual Machine start up parameters. For example Java EE applications, Spring based Web applications or simply your own home grown web application packaged in a .war file.

You may download or check out the source code at http://usemon.org

Prologue

Recently we experienced a problem regarding the RMI-compability between IBM WebSphere (WAS) version 5.1 and BEA WebLogic (WLS) version 8.3 both running a version 1.4.2 JVM. The WLS is a client of the WAS server and this worked fine since both the IBM and BEA RMI-implementation is based upon Sun’s RMI-ORB. Then an application came along that needed a custom login module and we had to enable security attribute propagation in WAS. All of a sudden hell broke loose, and the WLS was not able to speak with the WAS anymore. The problem was soon identified to be the wire format of the RMI-implementation. When you enable security attribute propagation, the wire format changes and WLS didn’t understand the WAS RMI response anymore.

The solution

In one way or the other I needed the RMI-implementation from WAS to work inside WLS. This is not an easy thing to do because a RMI-implementation register itself inside the JVM in several ways. I also wanted to hide the IBM classes from WLS to make sure I didn’t get any unexpected class ambiguities.

First I needed to just be able to call a EJB on a WAS server from a remote JVM. This was pretty easy and well documented from IBM. Since I didn’t need the support for DataSources or JMS resources I opted for the WebSphere pluggable client that only support EJB invocations. The following list includes the relevant WAS libraries needed to run the pluggable client.

"ibmext.jar", "ibmorb.jar", "iwsorbutil.jar", "naming.jar", "namingclient.jar", "properties.jar", "txClientPrivate.jar", "ras.jar", "wsexception.jar", "bootstrap.jar", "ecutils.jar", "iwsorb.jar", "ffdc.jar", "idl.jar", "txClient.jar"

The solution involved emulating the environment to let the IBM RMI-implementation believe it was running in a standard client JVM without knowing about BEA RMI and vica versa. I therefore created an IsolatingClassLoader that knew how to load the IBM client jars from the filesystem and not from the classpath. The same ClassLoader also knew what packages and classes it was supposed to isolate and to do the loading itself instead of delegating it to the parent ClassLoader.

  public class IsolatingClassLoader extends URLClassLoader {
    ...
    ...
    protected synchronized Class loadClass(String name, boolean resolve) throws ClassNotFoundException {
      boolean isolated = isolatedClassNames.contains(name);
      if (!isolated) {
        for (int i = 0; i < isolatedPackagePrefixes.length; i++) {
          if (name.startsWith(isolatedPackagePrefixes[i])) {
            isolated = true;
            break;
          }
        }
      }
      if (isolated) {
        Class c = findLoadedClass(name);
        if (c == null) c = findClass(name);
        if (resolve) resolveClass(c);
        return c;
      }
    return super.loadClass(name, resolve);
  }
}

By doing this I isolated all com.ibm. packages and classes. Before doing the lookups I had to substitute the context ClassLoader of the current thread to ensure that classes was loaded with the IsolatingClassLoader and register the thread to ensure the correct UtilDelegate is used (More on that below). This was restored and unregistered in a finally clause right after doing the lookup and narrowing.

ClassLoader oldClassLoader = Thread.currentThread().getContextClassLoader();
Thread.currentThread().setContextClassLoader(isolatingClassLoader);
ThreadRegistry.getInstance().register();
try {
  InitialContext ctx = new InitialContext(jndiConfig);
  Object o = ctx.lookup(”iiop://wasserverhostname:2809/cell/clusters/mycluster/ejb/com/path/to/ejb/SomeEJBHome”);
  SomeEJBHome home = (SomeEJBHome) PortableRemoteObject.narrow(o, SomeEJBHome.class);
  return (SomeEJBRemoteHome) Proxy.newProxyInstance(getClass().getClassLoader(), new Class[] \\
     { SomeEJBRemoteHome.class }, new IBMIvocationHandler(home.create(), isolatingClassLoader));
} finally {
  ThreadRegistry.getInstance().unregister();
  Thread.currentThread().setContextClassLoader(oldClassLoader);
}

The java.naming configuration also needed modification to use IBM’s implementation and using the constructor of the InitialContext class I supplied my own naming configuration.

As you can see from the lookup code I was forced to not return the actual RemoteHome object but actually a proxied version. Inside the IBMInvocationHandler I had to do the ContextClassLoader manipulation and thread registering again to ensure that IBM classes was available during the remote EJB invocation.

public class IBMIvocationHandler implements InvocationHandler {
  private ClassLoader isolatingClassLoader;
  private Object remoteStub;

  public MetroIvocationHandler(Object remoteStub, ClassLoader isolatingClassLoader) {
    this.remoteStub = remoteStub;
    this.isolatingClassLoader= isolatingClassLoader;
  }

  public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
    ClassLoader oldClassLoader = Thread.currentThread().getContextClassLoader();
    Thread.currentThread().setContextClassLoader(isolatingClassLoader);
    ThreadRegistry.getInstance().register();
    try {
      return method.invoke(remoteStub, args);
    } finally {
      ThreadRegistry.getInstance().unregister();
      Thread.currentThread().setContextClassLoader(oldClassLoader);
    }
  }
}

We have soon reached our goal, but one nasty obstacle is standing between ourselves and a solution. That is the javax.rmi.CORBA.Util class that holds a private reference to a singleton implementing the javax.rmi.CORBA.UtilDelegate inteface providing utility methods that can be used by stubs and ties to perform common operations. The implementation is defined by a system property (javax.rmi.CORBA.UtilClass) and is initialized during VM startup. We could simply have replaced the private field containting the UtilDelegate with IBM’s implementation (com.ibm.ws.orb.WSUtilDelegateImpl), but then BEA’s RMI-implementation would get into trouble and we would see some really unbelievable exceptions ;-). Instead I created a proxy version of the UtilDelegate that chooses between the original and IBM’s implementation depending on which RMI-implementation that was using it. The ThreadRegistry uses a WeakHashMap to store the current Thread to identify which Thread is coming through IBM code or BEA RMI code.

public class UtilDelegateInvocationHandler implements InvocationHandler {
  private UtilDelegate original;
  private UtilDelegate ibm;

  public UtilDelegateInvocationHandler(UtilDelegate original, UtilDelegate ibm) {
    this.original = original;
    this.ibm = ibm;
  }

  public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
    iif(ThreadRegistry.getInstance().isRegistered()) {
        return method.invoke(ibm, args);
      else
       return method.invoke(original, args);
  }

}

The proxied UtilDelegate was installed using the reflection API and required that Java2 Security allowed my class to modify the accessible flags.

UtilDelegate ibmUD = (UtilDelegate) \\
      isolatingClassLoader.loadClass("com.ibm.ws.orb.WSUtilDelegateImpl").newInstance();
Class c = Class.forName("javax.rmi.CORBA.Util");
Field ff = c.getDeclaredField("utilDelegate");
ff.setAccessible(true);
UtilDelegate originalUD = (UtilDelegate) ff.get(null);
UtilDelegate proxyUtilDelegate = (UtilDelegate) Proxy.newProxyInstance(getClass().getClassLoader(), \\
      new Class[] { UtilDelegate.class }, new UtilDelegateInvocationHandler(originalUD, ibmUD));
ff.set(null, proxyUtilDelegate);

Conclusion

After doing this we could successfully use both RMI-implementations interchangably inside the WLS server and finally invoke EJBs running inside a WAS cluster with the security attribute propagation enabled.

Source Code

You may download the complete source code: was-client-inside-wls-updated-src

Update: 2. May, 2008

After beeing challange by Asgeir Nilsen I came up with a much improved way of determining which UtilDelegate implementation to use. In the old version I parsed the stack trace, but in this new and improved version I register the Thread in a singleton instead. The source archive has been updated to reflect this change.