• View
• Edit
• History
• Print

JNA

Wikipedia says: "Java Native Access provides Java programs easy access to native shared libraries without using the Java Native Interface."

Below are notes about JNA and some issues I faced.

Resources

• https://github.com/twall/jna
• http://twall.github.com/jna/3.5.0/javadoc/
• http://tutortutor.ca/cgi-bin/makepage.cgi?/articles/rjna - Tutorial See Revisiting Callbacks
• https://github.com/twall/jna/blob/master/www/CallbacksAndClosures.md

JNA Debugging

• Java properties: Invoke java with these properties set
  • -Djna.boot.library.path=${PTII}/lib ? If set, then it should contain the path to the jnidispatch native shared library
    • "The jna.boot.library.path property is mainly to support jna.jar being included in -Xbootclasspath, where java.library.path and LD_LIBRARY_PATH are ignored. It is also useful for designating a version of the library to use in preference to any which may already be installed on the system."
    • JNA Native Class docs
    • JNA Javadoc overview summary Loading
  • -Djna.debug_load=true Print out info about how the file is found - JNA FAQ
  • -Djna.debug_load.jna Trace how JNA looks for its own native libraries (jnidispatch) - JNA FAQ
  • -Djna.dump_memory=true Cause toString() on a jna Native to also print the memory. Excellent for finding alignment issues. - JNA FAQ
  • -Djna.encoding
    • Native.getDefaultEncoding() returns the value of jna.encoding or Native.DEFAULT_ENCODING
  • -Djna.library.path
    • "The search path for loaded native libraries may be modified by setting jna.library.path and a few other properties. You may also bundle native libraries in a jar file and have JNA automatically extract them for loading. See NativeLibrary for details. "
    • JNA Javadoc overview summary
    • Native.getWebStartLibraryPath() docs mention jna.library.path
  • jna.noclasspath=true If set, then don't use the Java classpath to find libraries. JNA Native Class docs
  • -Djna.nosys=true If set, then System.loadLibrary() is not called when looking for the JNA native library. JNA Native Class docs
  • -Djna.nounpack=true. If set, avoid automatic unpacking of the native library (useful for forcing failure if the JNA native library is not setup properly. JNA Native Class docs
  • -Djna.tmpdir Native.getTempDir() mentions jna.tmpdir
• Java calls
  • Native.setProtected(true) Might prevent crashes JNA JavaDoc Overview Summary Crash Protection

Varags

JNA Varargs

• JNA 3.5.1 Varargs javadoc says

• Old Bug: JNA-70: Corruption passing Structure byvalue as varargs
• Mailing list (1/08/2010): Re: Direct mapping and varargs - JNA Direct Mapping notes We are not using JNA direct mapping yet.

• Mailing list (8/23/08): Callback does not handle String or varargs

• Stack Overflow processing va_list argument in JNA callback function (June, 2012
  "You'll have to look at the actual implementation of va_list on the platform in question (start with the header file (vararg.h), and possibly delving into compiler specifics). va_list is usually a pointer onto the stack, with the various extraction args simply offset operations that move the pointer along by the appropriate size offset. You'll have to extract the stuff manually using the Pointer, using the various data extraction methods like Pointer.getInt(int offset), depending on the type you're attempting to access. –"
• https://groups.google.com/forum/#!msg/jna-users/k-xidZOv04M/noanEPj7l98J Eric Chiu asking about JNA Varargs on the JNA list in November, 2012)
• RFC: variadic closures in x86/x86_64 in December, 2013
  • D. Hogan asked for feedback about modify libffi to support variadic callbacks
  • Andrew Haley replied: "Why do you not define a variadic C function which does this:"
    void f1(int n, ...) {
    va_list ap;
    
    jintArray argsArray = (*env_p)->NewIntArray(n);
    jint *args = (*env_p)->GetIntArrayElements(argsArray, NULL);
    
    int argno = 0;
    va_start(ap, n);
    while (argno < n) {
    args[argno++] = va_arg(ap, int);
    }
    va_end(ap);
    (*env_p)->ReleaseIntArrayElements(argsArray, args, 0);
    (*env_p)->CallStaticVoidMethod(clsH, printargs, argsArray);
    }
    

    [$[Get Code]]
    "to call this variadic Java code:"
    
    public static void printargs(int... args) {
    for (int i : args)
    System.out.print(i + " ");
    System.out.println();
    }
    

    [$[Get Code]]
    "There is no reason that JNA cannot use this mechanism, is there?"
  • D.Hogan replied: "I'm sure it would work. A similar suggestion is mentioned in the CFFI documentation. It would require a C function for every assumed variadic processing convention. For instance, FMI requires #<Type><valueReference># which is not a typical printf/logger convention. I didn't go down that path because I thought the manual was asking for patches for a different way."

Patching JNA to work with Varargs CallBacks

As the libffi maintainers don't seem interested in supporting the functionality we require, here are patching instructions

Resources

• jna_4.0_diff.txt - D. Hogan's patch against jna 4.0 (August 13, 2013) See below for how to use.
• jfmi_svn_varargs_diff.txt D. Hogan's patch against the Ptolemy tree for org/ptolemy/fmi (August 13, 2013). Applied to the tree and various updates to format processing have been done. For historical reference only. No need to patch
• https://github.com/dhogan1/libffi/tree/variadic_closures - D. Hogan's libffi tree from December, 2013. ''For reference)

Procedure

A branch with the variadic extensions may be found at https://github.com/cxbrooks/jna.

For each of the platforms, we need to create the appropriate native jar. See https://github.com/twall/jna/blob/master/www/ReleasingJNA.md.

For Mac and Linux, do:

For Windows, see https://github.com/twall/jna/blob/master/www/WindowsDevelopmentEnvironment.md and follow the steps above. I used Cygwin.

Below is how it was created.

Below are the older instructions from D. Hogan

Building JNA from sources

See Patching JNA to work with Varargs CallBacks above.

These instructions are for unpatched JNA trees.

Building JNA Under Linux

2015-02-23

Under RHEL 6.x with Java 1.8,

 git clone https://github.com/twall/jna.git jna.git
 cd jna.git
 ant

Failed with:

     [exec] gcc -m64 -W -Wall -Wno-unused -Wno-parentheses -fPIC  -O2 -fno-omit-frame-pointer -fno-strict-aliasing  -D_REENTRANT -DHAVE_PROTECTION -I"/usr/lib/jvm/jdk1.8.0_31/jre/../include" -I"/usr/lib/jvm/jdk1.8.0_31/jre/../include/linux" -I"/home/cxh/src/jna/jna.git/build/native-linux-x86-64" -I/home/cxh/src/jna/jna.git/build/native-linux-x86-64/libffi/include -DJNA_JNI_VERSION='"4.0.1"' -DCHECKSUM='"1a6047467b59e8748f975e03016ce3d9"' -c dispatch.c -o /home/cxh/src/jna/jna.git/build/native-linux-x86-64/dispatch.o
     [exec] In file included from dispatch.c:82:
     [exec] /usr/lib/jvm/jdk1.8.0_31/jre/../include/linux/jawt_md.h:29:22: error: X11/Xlib.h: No such file or directory
     [exec] /usr/lib/jvm/jdk1.8.0_31/jre/../include/linux/jawt_md.h:30:23: error: X11/Xutil.h: No such file or directory
     [exec] /usr/lib/jvm/jdk1.8.0_31/jre/../include/linux/jawt_md.h:31:27: error: X11/Intrinsic.h: No such file or directory
     [exec] In file included from dispatch.c:82:
     [exec] /usr/lib/jvm/jdk1.8.0_31/jre/../include/linux/jawt_md.h:43: error: expected specifier-qualifier-list before ‘Drawable’
     [exec] dispatch.c: In function ‘Java_com_sun_jna_Native_getWindowHandle0’:
     [exec] dispatch.c:3004: error: ‘JAWT_X11DrawingSurfaceInfo’ has no member named ‘drawable’
     [exec] make: *** [/home/cxh/src/jna/jna.git/build/native-linux-x86-64/dispatch.o] Error 1

Solution: Install X11 includes.

  sudo yum install libX11-devel.i686

ant still failing:

 native:
     [exec] gcc -m64 -W -Wall -Wno-unused -Wno-parentheses -fPIC  -O2 -fno-omit-frame-pointer -fno-strict-aliasing  -D_REENTRANT -DHAVE_PROTECTION -I"/usr/lib/jvm/jdk1.8.0_31/jre/../include" -I"/usr/lib/jvm/jdk1.8.0_31/jre/../include/linux" -I"/home/cxh/src/jna/jna.git/build/native-linux-x86-64" -I/home/cxh/src/jna/jna.git/build/native-linux-x86-64/libffi/include -DJNA_JNI_VERSION='"4.0.1"' -DCHECKSUM='"1a6047467b59e8748f975e03016ce3d9"' -c dispatch.c -o /home/cxh/src/jna/jna.git/build/native-linux-x86-64/dispatch.o
     [exec] In file included from dispatch.c:82:
     [exec] /usr/lib/jvm/jdk1.8.0_31/jre/../include/linux/jawt_md.h:31:27: error: X11/Intrinsic.h: No such file or directory
     [exec] make: *** [/home/cxh/src/jna/jna.git/build/native-linux-x86-64/dispatch.o] Error 1

Solution:

   sudo yum install libXt-devel.x86_64

Building JNA Under MacOSX with Java 1.7

I wanted to build JNA under MacOSX so that I could do some debugging by modifying JNA.

Sadly, Apple's Java story is rather lacking. I installed the Java 1.7 preview for Mac OS X and now I can't seem to compile code that uses jni.h.

When compiling jna, I get:

This is because jni.h is a link back on to itself:

I went to https://developer.apple.com/downloads/index.action downloaded "Java for Mac OS X 10.7 Developer Package" and installed it, and the problem was not solved.

It appears that the issue here is that Apple's Java 1.6 used the Mac OS X "Frameworks" directory layout and Java 1.7 from Oracle uses the Java directory layout. Personally, I prefer the Java layout, since Java is "write once, test everywhere". Having a different directory layout under Mac OS X made things more difficult.

I'm not sure how my directory structure was broken, but the directory Headers directory itself had mod times of February 16, 2012, which is probably when I upgraded to Mac OS X 10.7.3.

Apple is no longer supporting Java and it is really starting to show.

So, I decided to move forward with the Java 1.7 preview. I needed to remove the broken jni.h links, so I did:

Download the jna sources using git:

2. Run ant

Problem: ant fails with "checking whether the C compiler works... no"

(This problem did not occur when working with the JNA head on 2 Aug 2013)

What is happening here is that the Mac OS X Java 1.7.0_04-ea returns amd64 for various properties.

The solution is to do

Problem: "jni_md.h: No such file or directory"

(This problem did not occur when working with the JNA head on 2 Aug 2013) ant -DARCH=x86_64 fails with:

My solution was to edit jna/native/Makefile.

Change:

to

Rerun:

Problem: "‘JAWT_MacOSXDrawingSurfaceInfo’ undeclared"

(This problem did not occur when working with the JNA head on 2 Aug 2013)

See http://www.mail-archive.com/bsd-port-dev@openjdk.java.net/msg01291.html

Workaround: Edit jna/native/dispatch.c and search for JAWT_MacOSXDrawingSurfaceInfo. Change

to

Rerun:

That seemed to do it! jna.jar is created.

Test failures:

Not all the tests pass:

(This problem did not occur when working with the JNA head on 2 Aug 2013)

I'm not sure about the above.

(This problem did not occur when working with the JNA head on 2 Aug 2013)

Oddly, javaws is in my path?

Building a debug version of JNA

1. Edit build.xml and change

to

1. Edit native/Makefile and change

to

I also added

to the top.

Build with:

Running with gdb under Mac OS X

See Debugging A Mac FMU

Building JNA Under Windows

These notes are about building JNA from source without the Variadic extensions

• https://github.com/twall/jna/blob/master/www/WindowsDevelopmentEnvironment.md Windows Development Environment (from twall's github)

Building JNA Under Cygwin

With a complete installation of Cygwin64 under Windows Server 2012R2, building JNA under Cygwin worked out of the box for me.

See Windows Development Environment (from twall's github) for what Cygwin packages need to be installed.

However, the tests hung, see Running JNA Tests under Windows

Building JNA Under Microsoft Visual C

See Windows Development Environment (from twall's github) has a partial description of what is necessary, below are the changes that I had to make to build JNA from the git head on 2/24/2015:

Building using Open VS2012 x64 Native Tools Command

If you are not heavily using Cygwin, then try using the MSVC tools without Cygwin.

In Cygwin Bash, check out the tree:

Start up "Open VS2012 x64 Native Tools Command" and run

Open VS2012 x64 Native Tools Command problem: The build is done with gcc, not cl

The variables in Windows Development Environment (from twall's github) need to be set or else the build will occur with gcc.

24-Feb-2015: I stopped using Open VS2012 x64 here and focused on using Cygwin below

Building under Cygwin Bash with MSVC tools: Set up environment variables

Windows Development Environment (from twall's github) says "Sample configuration, setting up INCLUDE/LIB:" and then lists what looks like bash commands for setting variables.

1. Under Windows Server 2012R2, I don't have C:/Program Files/Microsoft SDKs/Windows/v7.1 I looked in to installing it and got no where, so I decided to move forward and see what happened.
2. When I adjusted the variables for my machine as follows:
   export MSVC=c:/Program\ Files\ \(x86\)/Microsoft\ Visual\ Studio\ 11.0/vc
   export WSDK=c:/Program\ Files\ \(x86\)/Microsoft\ SDKs/Windows/v7.0A
   export WSDK_64=c:/Program\ Files\ \(x86\)/Microsoft\ SDKs/Windows/v7.0A
   export INCLUDE="$(cygpath -m "$MSVC")/include;$(cygpath -m "$WSDK")/include"
   export LIB="$(cygpath -m "$MSVC")/lib/amd64;$(cygpath -m "$WSDK")/include"
   

   [$[Get Code]]
   and then ran ant, then cl was not found.
3. To get the proper variables, I started up "Open VS2012 x64 Native Tools Command" and ran
   env > env.out
   

   [$[Get Code]]
   NOTE You are better off running ant in "Open VS2012 x64 Native Tools Commands" than using Cygwin bash.
4. Optional: If you really want to use Cygwin bash, then follow this step' I then added the variables that contained the word microsoft to a script and made the following edits
   1. Changed backslashes to forward slashes
   2. Modified each line so that it because with export=" and ended with a double quote: "
5. The resulting script is available as setupmsvc.txt and looks like
   export MSVC=c:/Program\ Files\ \(x86\)/Microsoft\ Visual\ Studio\ 11.0/vc
   export WSDK=c:/Program\ Files\ \(x86\)/Microsoft\ SDKs/Windows/v7.0A
   export WSDK_64=c:/Program\ Files\ \(x86\)/Microsoft\ SDKs/Windows/v7.0A
   export INCLUDE="$(cygpath -m "$MSVC")/include;$(cygpath -m "$WSDK")/include"
   export LIB="$(cygpath -m "$MSVC")/lib/amd64;$(cygpath -m "$WSDK")/include"
   
   export ExtensionSdkDir="C:/Program Files (x86)/Microsoft SDKs/Windows/v8.0/ExtensionSDKs"
   export FrameworkDir="C:/Windows/Microsoft.NET/Framework64"
   export FrameworkDIR64="C:/Windows/Microsoft.NET/Framework64"
   export FrameworkVersion="v4.0.30319"
   export FrameworkVersion64="v4.0.30319"
   export FSHARPINSTALLDIR="C:/Program Files (x86)/Microsoft SDKs/F#/3.0/Framework/v4.0/"
   
   export INCLUDE="C:/Program Files (x86)/Microsoft Visual Studio 11.0/VC/INCLUDE;C:/Program Files (x86)/Microsoft Visual Studio 11.0/VC/ATLMFC/INCL\
   UDE;C:/Program Files (x86)/Windows Kits/8.0/include/shared;C:/Program Files (x86)/Windows Kits/8.0/include/um;C:/Program Files (x86)/Windows Kits\
   /8.0/include/winrt;"
   
   export LIB="C:/Program Files (x86)/Microsoft Visual Studio 11.0/VC/LIB/amd64;C:/Program Files (x86)/Microsoft Visual Studio 11.0/VC/ATLMFC/LIB/am\
   d64;C:/Program Files (x86)/Windows Kits/8.0/lib/win8/um/x64;"
   
   export LIBPATH="C:/Windows/Microsoft.NET/Framework64/v4.0.30319;C:/Windows/Microsoft.NET/Framework64/v3.5;C:/Program Files (x86)/Microsoft Visual\
   Studio 11.0/VC/LIB/amd64;C:/Program Files (x86)/Microsoft Visual Studio 11.0/VC/ATLMFC/LIB/amd64;C:/Program Files (x86)/Windows Kits/8.0/Referen\
   ces/CommonConfiguration/Neutral;C:/Program Files (x86)/Microsoft SDKs/Windows/v8.0/ExtensionSDKs/Microsoft.VCLibs/11.0/References/CommonConfigura\
   tion/neutral;"
   
   export PATH="/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 11.0/Common7/IDE/CommonExtensions/Microsoft/TestWindow:/cygdrive/c/Program F\
   iles (x86)/Microsoft Visual Studio 11.0/VC/BIN/amd64:/cygdrive/c/Windows/Microsoft.NET/Framework64/v4.0.30319:/cygdrive/c/Windows/Microsoft.NET/F\
   ramework64/v3.5:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 11.0/VC/VCPackages:/cygdrive/c/Program Files (x86)/Microsoft Visual Studi\
   o 11.0/Common7/IDE:/cygdrive/c/Program Files (x86)/Microsoft Visual Studio 11.0/Common7/Tools:/cygdrive/c/Program Files (x86)/HTML Help Workshop:\
   /cygdrive/c/Program Files (x86)/Microsoft Visual Studio 11.0/Team Tools/Performance Tools/x64:/cygdrive/c/Program Files (x86)/Microsoft Visual St\
   udio 11.0/Team Tools/Performance Tools:/cygdrive/c/Program Files (x86)/Windows Kits/8.0/bin/x64:/cygdrive/c/Program Files (x86)/Windows Kits/8.0/\
   bin/x86:/cygdrive/c/Program Files (x86)/Microsoft SDKs/Windows/v8.0A/bin/NETFX 4.0 Tools/x64:/cygdrive/c/Program Files (x86)/Microsoft SDKs/Windo\
   ws/v7.0A/Bin/x64:/cygdrive/c/Program Files (x86)/Microsoft SDKs/Windows/v8.0A/bin/NETFX 4.0 Tools:/cygdrive/c/Program Files (x86)/Microsoft SDKs/\
   Windows/v7.0A/Bin:/usr/bin:/usr/local/apache-ant-1.9.4/bin:/cygdrive/c/Program Files/Java/jdk1.8.0_31/bin:/cygdrive/c/ProgramData/Oracle/Java/jav\
   apath:/cygdrive/c/Windows/system32:/cygdrive/c/Windows:/cygdrive/c/Windows/System32/Wbem:/cygdrive/c/Windows/System32/WindowsPowerShell/v1.0:/cyg\
   drive/c/Program Files/MATLAB/R2014b/runtime/win64:/cygdrive/c/Program Files/MATLAB/R2014b/bin:/cygdrive/c/Program Files/MATLAB/R2014b/bin/win64:/\
   cygdrive/c/Program Files (x86)/Microsoft ASP.NET/ASP.NET Web Pages/v1.0:/cygdrive/c/Program Files/Microsoft SQL Server/110/Tools/Binn:/cygdrive/c\
   /Program Files/Microsoft/Web Platform Installer:/cygdrive/c/Program Files/TortoiseSVN/bin"
   
   export PSModulePath="C:/Windows/system32/WindowsPowerShell/v1.0/Modules/;c:/Program Files/Microsoft Security Client/MpProvider/"
   
   export WindowsSdkDir="C:/Program Files (x86)/Windows Kits/8.0/"
   export WindowsSdkDir_35="C:/Program Files (x86)/Microsoft SDKs/Windows/v7.0A/Bin/"
   export WindowsSdkDir_old="C:/Program Files (x86)/Microsoft SDKs/Windows/v8.0A/"
   
   export USE_MSVC=1
   

   [$[Get Code]]
   Note that I also appended export USE_MSVC= because native/Makefile says:
   # The windows build requires a cygwin or MSYS installation. The build may use
   # MS tools or GCC/mingw. Define USE_MSVC in your environment and ensure cl.exe
   # and ml.exe are in your PATH to build with MS tools.
   

   [$[Get Code]]
6. In Cygwin bash, to set the variables, use the dot command:
   . setupmsvc.txt
   

   [$[Get Code]]
7. In Cygwin bash, check out the tree and run ant
   git clone https://github.com/twall/jna.git
   cd jna.git
   ant
   

   [$[Get Code]]

Various issues are discussed below

invalid syntax in conditional

While running ant:

  [exec] Makefile:165: *** invalid syntax in conditional. Stop

Line 165 of native/Makefile is the ifdef $(MSVC) line below:

  ifdef $(MSVC)
  USE_MSVC=true
  endif

I had export MSVC=1, so a workaround was:

  ifneq ($(MSVC), "")
  USE_MSVC=true
  endif

I think the issue here is that MSVC is set to a value that has spaces in it?

Cygwin with MSVC: configure: error: C compiler cannot create executables

When building the from the variadic extensions git repo under Cygwin with MSVC:

Looking at jna-cxbrooks/build/native-win32-x86-64/libffi/config.log:

The solution is to edit jna-cxbrooks/native/Makefile and change:

  FFI_ENV=CC="$(CC)" CFLAGS="$(COPT) $(CDEBUG) -std=gnu99" CPPFLAGS="$(CDEFINES)"

to:

  FFI_ENV=CC="$(CC)" CFLAGS="$(COPT) $(CDEBUG)" CPPFLAGS="$(CDEFINES)"

and rerun ant

Note that the above issue does 'not occur in the jna head as of February, 2015. This problem is probably in the 4.1 jna branch.

Cygwin with MSVC: Unsupported argument '-std=gnu99'

When building the from the variadic extensions git repo under Cygwin with MSVC:

The solution is to edit jna-cxbrooks/native/Makefile and change:

  PCFLAGS=-W -std=gnu99 -Wall -Wno-unused -Wno-parentheses

to:

  PCFLAGS=-W  -Wall -Wno-unused -Wno-parentheses

Note that the above issue does 'not occur in the jna head as of February, 2015. This problem is probably in the 4.1 jna branch.

Cygwin with MSVC: unresolved external symbol ffi_closure_va_sint8

When building the from the variadic extensions git repo under Cygwin with MSVC:

Note that the above issue does 'not occur in the jna head as of February, 2015. This problem is probably with the variadic patch.

Analysis:

The missing symbols are defined in jna-cxbrooks/native/libffi/src/closures_va.c. Running nm on the .obj file indicates that they are not present. Looking at closure_va.c, we see

The issue here is that _STDC_VERSION is probably not defined for MSVC, so we change:

  #  if __STDC_VERSION__ >= 199901L

To:

  #  if __STDC_VERSION__ >= 199901L || defined(_MSC_VER)

This causes:

Cannot open ../build/native-win32-x86-64/testlib.dll

The problem here was that the .dll already existed. The solution was to run rm ./build/native-win32-x86-64/testlib.dll

Running ant clean did not fix this.

After this, build succeeded, though the tests failed, see Running JNA Tests under Windows

Running JNA Tests under Windows

Running the JNA Tests using Eclipse under Windows

Running the tests using Eclipse under Windows worked better.

1. Create a project
   1. Right Click in the Package Explorer, select Import
   2. General -> Existing Projects into Workspace -> Next
   3. In the "Import Properties" Window, next to "Select root directory", click Browse. Browse to the source tree checked out with git. Click Finish.
2. Set up run configuration
   1. In the "Test" tab, select "Run all tests in the selected project, package or source folder."
   2. In the "Arguments" tab, under "VM arguments", enter
      -Ddummy
      -Djna.library.path=C:\cygwin64\home\cxh\src\jna\jna.git\build\native-win32-x86-64
      -Djna.nosys=true
      -Djna.protected=true
      -Djna.builddir=C:\cygwin64\home\cxh\src\jna\jna.git\build
      (:source:)
      # In the "Classpath" tab, Click on "User Entries", then "Add Jars" and browse to the @@build@@ directory and add @@jna.jar@@. Also, find the @@lib/test@@ directory and add all those jars.
      # The WebStartTest test fails, so:
      ## edit that file
      ## Comment out the body
      ## Add (:source:)
      public void testNotWebStart() {
      System.out.println("WebStartTest commented out.");
      }
      

      [$[Get Code]]
3. Click on Run

testBatchCheckStructuredGetFieldOrder

Fails with

  java.lang.NoClassDefFoundError: com/google/common/base/Predicate

Solution: Add lib/test/guava-11.0.2.jar to the classpath for the Run configuration. In fact, add all the jars in that directory and avoid other problems.

testLoadFromUnicodePath errors from within Eclipse under Windows

When running all the tests with the JNA head from 24-Feb-2015, JNALoadTest.testLoadFromUnicodePath() fails:

I'm not sure about the above, I'll skip it for now.

DirectTest testGetOptionsFOrDirectMappingWithMemberInitializer() fails

In the JNA head from 24-Feb-2015, this test fails from within Eclipse, under Windows:

DirectTest.testGetOptionsForDirectMappingWithStaticInitializer fails from within Eclipse under Windows

Native library not found

While testing the varargs extensions, win32-x86-64/testlib-path.dll was not found

The solution is to run ant test to build the test dlls.

Also, it turns out that the error was that I had the wrong -D values for the VM arugments - the values referred to a different directory.

Running JNA Tests using Cygwin under Windows

In the devel tree (no variadic extensions), the JNA Callback tests hang when I build from Cygwin using either gcc or MSVC. However, if I build from the Open VS2012 x64 Native Tools Command Prompt, then they don't hang. 24-Feb-2015: I could not replicate the previous statement, I was not able to build from the Open VS2012 x64 Native Tools Command Prompt. Building using Cygwin with MSVC and then running the tests from that tool also hung.

• Callbacks unit test failures (Github JNA Issues)

To replicate outside of ant, I ran ant -v and then pasted the output into a script file. Below is an example

jvisualvm does not show the process?

JNA Tests under Windows Server 2008R2

On Joule, which is running Windows Server 2008R2, the variadic tests failed in the jna devel tree on 2014-12-14.

    [junit] Testcase: testDLLCallback(com.sun.jna.CallbacksTest):       Caused an ERROR
    [junit] No more DLL callback slots available
    [junit]     at com.sun.jna.Native.createNativeCallback(Native Method)
    [junit]     at com.sun.jna.CallbackReference.<init>(CallbackReference.java:232)
    [junit]     at com.sun.jna.CallbackReference.getFunctionPointer(CallbackReference.java:398)
    [junit]     at com.sun.jna.CallbackReference.getFunctionPointer(CallbackReference.java:380)
    [junit]     at com.sun.jna.Function.convertArgument(Function.java:534)
    [junit]     at com.sun.jna.Function.invoke(Function.java:297)
    [junit]     at com.sun.jna.Library$Handler.invoke(Library.java:212)
    [junit]     at com.sun.proxy.$Proxy0.callVoidCallback(Unknown Source)
    [junit]     at com.sun.jna.CallbacksTest.testDLLCallback(CallbacksTest.java:1257)

• The above looks bad, need to look into it.
• Note that I get the same above failure when I check out the JNA 4.0 branch without variadic extensions.

    [junit] Testsuite: com.sun.jna.JNALoadTest
    [junit] Tests run: 5, Failures: 0, Errors: 1, Time elapsed: 4.852 sec
    [junit]
    [junit] Testcase: testLoadFromUnicodePath(com.sun.jna.JNALoadTest): Caused an ERROR
    [junit] JVM error: System.load() failed to load JNA native library from C:\Users\cxh.EECS\testLoadFromUnicodePath-?????\jna1665291666603896836.dll): java.lang.UnsatisfiedLinkError: C:\Users\cxh.EECS\testLoadFromUnicodePath-?????\jna1665291666603896836.dll: Can't find dependent libraries
    [junit] java.lang.Error: JVM error: System.load() failed to load JNA native library from C:\Users\cxh.EECS\testLoadFromUnicodePath-?????\jna1665291666603896836.dll): java.lang.UnsatisfiedLinkError: C:\Users\cxh.EECS\testLoadFromUnicodePath-?????\jna1665291666603896836.dll: Can't find dependent libraries
    [junit]     at com.sun.jna.JNALoadTest.testLoadFromUnicodePath(JNALoadTest.java:229)

• This might be common to all Windows installations.
• Note that I get the same above failure when I check out the JNA 4.0 branch without variadic extensions.

    [junit] Testcase: testLoadJAWT(com.sun.jna.LibraryLoadTest):        Caused an ERROR
    [junit] The specified module could not be found.
    [junit]
    [junit] java.lang.UnsatisfiedLinkError: The specified module could not be found.
    [junit]
    [junit]     at com.sun.jna.Native.getWindowHandle0(Native Method)
    [junit]     at com.sun.jna.Native$AWT.getComponentID(Native.java:2010)
    [junit]     at com.sun.jna.Native$AWT.getWindowID(Native.java:1986)
    [junit]     at com.sun.jna.Native.getWindowPointer(Native.java:275)
    [junit]     at com.sun.jna.LibraryLoadTest$AWT.loadJAWT(LibraryLoadTest.java:286)
    [junit]     at com.sun.jna.LibraryLoadTest.testLoadJAWT(LibraryLoadTest.java:49)

The above is expected because I'm connected via ssh with no X11 setup. This should be verified by running on the machine.

   [junit] Testsuite: com.sun.jna.WebStartTest
    [junit] Tests run: 6, Failures: 0, Errors: 6, Time elapsed: 181.583 sec
    [junit]
    [junit] Testcase: testDetectError(com.sun.jna.WebStartTest):        Caused an ERROR
    [junit] JWS Timed out
    [junit] java.lang.Error: JWS Timed out
    [junit]     at com.sun.jna.WebStartTest.runTestUnderWebStart(WebStartTest.java:199)
    [junit]     at com.sun.jna.WebStartTest.runTestUnderWebStart(WebStartTest.java:240)
    [junit]     at com.sun.jna.WebStartTest.runBare(WebStartTest.java:373)

The above can probably be ignored.

BTW - Java 1.8 or so now requires setup for the Java Web Start local files to work. Under Windows, start up the Java Control Panel and add file:/ as an acceptable host.

Notes about building JNA-4.0 under Windows

• See https://github.com/twall/jna/blob/master/www/WindowsDevelopmentEnvironment.md

To get build JNA-4.0 from sources under Windows Server 2012R2 64-bit, I installed Visual Studio. In the Windows Environment Variables control panel, I set

• ANT_HOME
• JAVA_HOME
• PATH to include both ANT_HOME/bin and JAVA_HOME/bin

1. Start up the Open VS2012x74 Native Tools Command, which will sent the environment to use the cl Visual Studio Compiler
2. Download the jna tree:
   git clone https://github.com/twall/jna.git jna.git
   cd jna.git
   git checkout -b 4.0-test 4.0
   

   [$[Get Code]]
3. Build:
   ant
   

   [$[Get Code]]

JNA Notes

Random notes and trains of thoughts...

JNA Struct By Reference

http://www.eshayne.com/jnaex/example03.html C:

Java:

JNA Invalid Memory access

When loading a 64-bit FMU created by Dymola, the following was observed:

  Caused by: java.lang.Error: Invalid memory access
    at com.sun.jna.Native.invokePointer(Native Method)
    at com.sun.jna.Function.invokePointer(Function.java:470)
    at com.sun.jna.Function.invoke(Function.java:404)
    at com.sun.jna.Function.invoke(Function.java:315)
    at com.sun.jna.Function.invoke(Function.java:268)
    at ptolemy.actor.lib.fmi.FMUImport.preinitialize(FMUImport.java:1861)

Line 1861 is:

                _fmiComponent = (Pointer) _fmiInstantiateFunction.invoke(
                        Pointer.class, new Object[] { getFullName(), fmiType,
                            _fmiModelDescription.guid,
                            _fmiModelDescription.fmuResourceLocation,
                            _callbacks, toBeVisibleFMI2, loggingOnFMI2 });

The function being called is

   /* Creation and destruction of FMU instances and setting debug status */
   typedef fmi2Component fmi2InstantiateTYPE (fmi2String, fmi2Type, fmi2String, fmi2String, const fmi2CallbackFunctions*, fmi2Boolean, fmi2Boolean);

with these typedefs and structs:

   typedef void*           fmi2Component;               /* Pointer to FMU instance       */
   typedef void*           fmi2ComponentEnvironment;    /* Pointer to FMU environment    */
   typedef void*           fmi2FMUstate;                /* Pointer to internal FMU state */
   typedef unsigned int    fmi2ValueReference;
   typedef double          fmi2Real   ;

   typedef int             fmi2Integer;
   typedef int             fmi2Boolean;
   typedef char            fmi2Char;
   typedef const fmi2Char* fmi2String;
   typedef char            fmi2Byte;

  typedef enum {
    fmi2ModelExchange,
    fmi2CoSimulation
  } fmi2Type;

  typedef void      (*fmi2CallbackLogger)        (fmi2ComponentEnvironment, fmi2String, fmi2Status,   fmi2String, fmi2String, ...);
   typedef void*     (*fmi2CallbackAllocateMemory)(size_t, size_t);
  typedef void      (*fmi2CallbackFreeMemory)    (void*);
  typedef void      (*fmi2StepFinished)          (fmi2ComponentEnvironment, fmi2Status);

  typedef struct {
   const fmi2CallbackLogger         logger;
   const fmi2CallbackAllocateMemory allocateMemory;
   const fmi2CallbackFreeMemory     freeMemory;
   const fmi2StepFinished           stepFinished;
   const fmi2ComponentEnvironment   componentEnvironment;
  } fmi2CallbackFunctions;

Interestingly, running without the variadic extensions fails with both jna-4.0.0.jar and jna-4.1.0.jar:

  java -classpath "c:\Users\cxh\ptII\lib\jna-4.0.0.jar;c:\Users\cxh\ptII" ptolemy.vergil.VergilApplication -ptiny *.xml

Also, using the fmusdk fmus compiled with Microsoft Visual Studio also works:

  java -classpath "c:\Users\cxh\ptII\lib\jna-4.1.0-variadic.jar;c:\Users\cxh\ptII"      org.ptolemy.fmi.driver.FMUCoSimulation      c:/Users/cxh/src/fmi/fmusdk/fmu20/fmu/cs/x64/inc.fmu 11.0 1.0 true

Disassembly of the Dymola fmi2Instantiate method.

Running gdb on the fmusdk/bin/x64/fmu20sim_cs.exe

Running on a fmusdk 64-bit fmu compiled with MSVC:

Interestingly, comparing the two versions, they are the same until the 5th line.

Why are they different? Probably because inc's fmi2Instantiate has a local (ModelInstance *comp) on the stack. I'm not sure why they are 16 bytes different. Could be alignment.

In the 64-bit inc.fmu, fmi2Instantiate is defined in fmuTemplate.c:

  // ---------------------------------------------------------------------------
  // FMI functions
  // ---------------------------------------------------------------------------
  fmi2Component fmi2Instantiate(fmi2String instanceName, fmi2Type fmuType, fmi2String fmuGUID,
                            fmi2String fmuResourceLocation, const fmi2CallbackFunctions *functions,
                            fmi2Boolean visible, fmi2Boolean loggingOn) {
    // ignoring arguments: fmuResourceLocation, visible
    ModelInstance *comp;
    if (!functions->logger) {
        return NULL;
    }

Resources:

• Invalid memory access (Stack Overflow) - Problem is with characters. Uses JNI, not JNA.
• java jna probleme Structure invalid access memory (Stack Overflow) Problem same as next resource
• Char** in java JNA #313 (Github JNA Issues)
• Java interoperation with a Native DLL using JNA (Viaboxx) - nice overview of JNA.
  • "Create Java objects from array of C-structs:
    • "We map the return type of the function as a com.sun.jna.Pointer in Java. With the knowledge about the C-struct (sequence and data type of member variables), the java program can read and convert the memory content to create adequate objects. The number of elements in the array is also known (variable “numberOfResults”, see above). It is very important, to compute the offset correctly to avoid JVM-crashes for invalid memory access!"

JNA Alignment

• http://today.java.net/article/2009/11/11/simplify-native-code-access-jna - article about JNA and Alignment
• JNA 3.5.1 Native javadoc -

• Re: [jna-users] setting structure alignment

fmiCallbackLogger

The fmiCallbackLogger() function is a callback in the fmiFunctions.h in the FMUSDK:

fmiPlatformTypes.h defines fmiComponent etc:

• The jnaerator provided me with code that works:

FMILibrary.java:

FMUCoSimulation.java:

However, when we run, we get various segfaults when the logger is invoked and we have the varargs arguments.

It turns out that the segfault problem was because the code was not properly allocating memory and keeping handles to the allocated memory. Thus, the allocated memory was being gc'd. The code in question has callbacks for allocating and freeing memory. The fix is to keep a list of allocated memory and have the allocate callback add a reference to the Java list and have the free callback remove the reference.

The code below more accurately represents the C code. FMILibrary.java:

FMUCoSimulation.java:

• Tried using Object ... parameters or Pointer ... parameters:

After modifying jna/src/com/sun/jna/Structure.java so that the exception thrown includes the cause and updating $PTII/lib/jna.jar, I get:

Timothy Wall commented:

"See the "caused" by exception, which causes the Callback field write to fail. JNA supports Callback (function pointer) fields within a structure, but in your case, one of the callback arguments is Object, which is not allowed (there is a system property "jna.allow_objects" or something similar intended to bypass that, but it's not thoroughly tested in all situations where an Object might be used).

"In general, JNA doesn't allow you to pass Java object references to native code, since that raises all sorts of reference tracking/GC issues."

JNA Custom Type Conversion

Search for "requires custom type conversion" brings up:

• Mailing list (08/07/08): Help with:Callback argument class [I requires custom type conversion
• Mailing list (7/24/09): Callbacks in structures

This is incomplete

I was able to create a custom type conversion called FMITypeMapper.java. See jna/src/com/sun/jna/win32/W32APITypeMapper.java. To include the mapper, I added the following to FMILibrary.java. This needed to appear in the Library interface class.