• View
• Edit
• History
• Print

GDPJS

(redirected from Modules.GDPJS)

The GDP is no longer supported by the Accessors Framework. This page exists as a historical reference.

This page is about using the GDP with JavaScript

How to use the GDP Accessors with Node.

Installation

To use the shared libraries in the GDP JavaScript interface, you must have certain packages installed.

If you are working from the Accessor svn repository, then run:

  accessors/web/gdp/adm/gdp-setup.sh

If you installed the @terraswarm/gdp npm module, then run:

  node_modules/@terraswarm/gdp/adm/gdp-setup.sh

What works

As of 8/22/16

• The following accessors work under Node and Cape Code:
  • GDPLogCreate - Create a GDP Log
  • GDPLogRead - One simple test - Read from a Log
  • GDPLogAppend - Append to a Log
  • GDPLogSubscribe - Subscribe to a Log
• A Cape Code model that reads from the GDP can generate a Composite Accessor that can be run in Node.
• @terraswarm/gdp is a Node npm module that includes a client interface to the GDP,
  • It is available via the Node Package Manager (npm) for Mac OS X and Linux, to install it, run
    npm install @terraswarm/gdp

    [$[Get Code]]

See gdp/lang/js/README.md for how to update the module.

What is missing

• More tests

Testing

The Node GDP accessor tests are based on CapeCode GDP Accessor tests, see CapeCode and the GDP

The CapeCode Code Generator is run on the CapeCode .xml tests to generate tests in accessors/gdp/test/auto/.

To run the tests,

1. Download the accessors repository
2. Install the @terraswarm/gdp module:
   ant terraswarm-gdp-update

   [$[Get Code]]
3. Install libraries necessary for the GDP (See Easier GDP Deployment: shared library issues)
   ./node_modules/@terraswarm/gdp/adm/gdp-setup.sh

   [$[Get Code]]
4. Run the tests:
   mocha   hosts/nod/test/testNodeOneAuto.js --Dauto=gdp/test/auto

   [$[Get Code]]

See the individual files in accessors/gdp/test/auto/ for how to run the CapeCode models and how to run the generated code using Node.

Cape Code Example

A Cape Code Model that reads from a log. When it is run, it uses the Java/GDP interface and produces the following on stdout:

  TestDisplay: foo (Thread[GDPLogRead,1,main])

Double clicking on the Blue AccessorCodeGenerator icon and then clicking on the generator button generates the following CompositeAccessor:

After clicking on generate, the dialog box contains:

The key thing is that the line

  foo

appears

Cape Code GDP Model to Node

Below are the steps necessary to create a Composite Accessor that reads a GDP record

Install Cape Code and the GDP

1. Download and install the development version of Ptolemy II, which includes the Cape Code configuration
   svn co https://repo.eecs.berkeley.edu/svn-anon/projects/eal/ptII/trunk ptII
   cd ptII
   export PTII=`pwd`
   ./configure
   ant
   (cd bin; make)

   [$[Get Code]]
2. Get access to the gdp git repository by requesting access from Eric.
3. Download the GDP. Installing the GDP in $PTII/vendors/gdp/gdp is preferred. (Note that the setupAndRun.sh script downloads the gdp_router into the directory above the gdp/ directory, thus, if $PTII/vendors/gdp/gdp_router will be created):
   cd $PTII/vendors
   mkdir gdp
   cd gdp
   git clone https://repo.eecs.berkeley.edu/git/projects/swarmlab/gdp.git

   [$[Get Code]]
4. Build the GDP. see gdp/README.md for installing the prerequisites.
   cd gdp
   make

   [$[Get Code]]

Start the GDP Daemons and initialize the log

1. While in the gdp directory, download the gdp_router and start it by using the script. Note that you may be prompted for your git password, so run the script in the foreground and then put it in the background.
   bash-3.2$ ./test/setupAndRun.sh
   #### ./test/setupAndRun.sh: Running git pull in /Users/cxh/ptII/vendors/gdp/gdp_router
   fatal: unable to access 'https://repo.eecs.berkeley.edu/git/projects/swarmlab/gdp_router.git/': Could not resolve host: repo.eecs.berkeley.edu
   #### ./test/setupAndRun.sh: Removing /Users/cxh/ptII/vendors/gdp/gcls and then recreating it.
   #### ./test/setupAndRun.sh: Set up /Users/cxh/ptII/vendors/gdp/ep_adm_params
    To run with these settings from the command line, use:
       export EP_PARAM_PATH=/Users/cxh/ptII/vendors/gdp/ep_adm_params
   #### ./test/setupAndRun.sh: Starting gdp_router
   Command to start gdp_router (cd /Users/cxh/ptII/vendors/gdp/gdp_router; python ./src/gdp_router.py -l routerLog.txt) &
   #### ./test/setupAndRun.sh: Starting gdplogd
   Command to start gdplogd: /Users/cxh/ptII/vendors/gdp/gdp/gdplogd/gdplogd -F -N ealmac23.local &
   My GDP routing name = ealmac23.local
   #### Creating log gdp.runTests.28866
   Command to create a log: /Users/cxh/ptII/vendors/gdp/gdp/apps/gcl-create -k none -s ealmac23.local gdp.runTests.28866
   Created new GCL aI-qVnWLmvJXGldKDgDoeHOnLVWAYOBdqNIP7jHlewQ
           on log server ealmac23.local
   exiting with status OK
   ./test/setupAndRun.sh: called with no arguments, gdp daemons running:
   cxh             20795   0.1  0.1  2496032  22608 s003  S+    7:51AM   0:00.15 /opt/local/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python ./src/gdp_router.py -l routerLog.txt
   cxh             20808   0.0  0.0  2445080    824 s003  S+    7:51AM   0:00.00 grep gdp
   cxh             20799   0.0  0.0  2459344   2104 s003  S+    7:51AM   0:00.01 /Users/cxh/ptII/vendors/gdp/gdp/gdplogd/gdplogd -F -N ealmac23.local
   ./test/setupAndRun.sh: Sleeping...
     C-c C-z
   [1]+  Stopped                 ./test/setupAndRun.sh
   bash-3.2$ bg
   [1]+ ./test/setupAndRun.sh &
   bash-3.2$

   [$[Get Code]]
2. While in the gdp/ directory, create the log named GDPLogRead1 (The log name is also used in the Cape Code model and the Composite Accessor)
   bash-3.2$ ./apps/gcl-create -k none -s `hostname` GDPLogRead1
   Created new GCL hU0fPyiI277oZ3knIPvzBYnTKEQ3EiMbuVjN6C_2yk8
           on log server ealmac23.local
   exiting with status OK
   bash-3.2$

   [$[Get Code]]
3. Add a record to the log:
   # Add a record to the log: (:source lang=bash:)
   bash-3.2$ echo "foo" | ./apps/gdp-writer -a GDPLogRead1
   GDPname: hU0fPyiI277oZ3knIPvzBYnTKEQ3EiMbuVjN6C_2yk8 (0 recs)
   
   Starting to read input
   Got input «foo»
   Asynchronous event type Asynchronous Status:
           recno 1, stat OK
   gdp-writer: exiting with status OK
   bash-3.2$

   [$[Get Code]]
4. For testing purposes, use the C-based binary to read from the log:
   bash-3.2$ ./apps/gdp-reader GDPLogRead1
   Reading GCL hU0fPyiI277oZ3knIPvzBYnTKEQ3EiMbuVjN6C_2yk8
    >>> recno 1, len 3, ts 2016-07-19T15:12:19.949130000Z
   00000000 66 6f 6f
            f  o  o
   exiting after 1 records with status WARNING: end of file [EPLIB:generic:3]
   bash-3.2$

   [$[Get Code]]

As an alternative, if you are running linux (tested on ubuntu), you can also use a docker setup (requires bridge-utils and docker) to have your own log-server and router. The advantage of this is that you can have multiple instances of log-servers/routers, and everything is contained in the docker images. For details, see:

• https://gdp.cs.berkeley.edu/redmine/projects/gdp/repository/revisions/master/entry/adm/docker-net/README.md (Non-public)
• gdp/adm/docker-net/README.md (in the gdp git repo)

Build the GDP/JavaScript interface

1. Build the GDP/JavaScript interface:
   cd lang/js/gdpjs
   make

   [$[Get Code]]

Install the GDP/JavaScript interface

The JavaScript interface to the GDP is located in gdp/lang/js. However, for Node to find it, we need to install the interface as a module named @terraswarm/gdp.

Modules may be installed either globally, which requires access to the root account, or locally. We install the module locally.

To refer to our module, the JavaScript file contains:

When node reads this line, it searches up the directory hierarchy for directories named node_modules/@terraswarm/gdp/.

To install the interface, we use the npm install command, which will create the node_modules directory.

1. In the near term, we install the module from the source files in the gdp repo. For our purposes, we run npm install in $PTII so that when we run node in $PTII/org.terraswarm/accessor/accessors/web/gdp/test/auto, the $PTII/node_modules directory is found:
   cd $PTII
   npm install vendors/gdp/gdp/lang/js

   [$[Get Code]]

Note that a number of packages will be installed.

The above will create $PTII/node_modules/@terraswarm/gdp/:

You may ignore messages like:

  npm WARN enoent ENOENT: no such file or directory, open '/Users/cxh/ptII/package.json'

If you want to get rid of those messages, create $PTII/package.json:

CapeCode

1. Up until now, we have been using the C-based binaries. The next step is to use the Java interface to the GDP by running Cape Code and to then generate a Composite Accessor and invoke it using Node
2. Start Cape Code:
   $PTII/bin/vergil -capecode $PTII/ptolemy/actor/lib/jjs/modules/gdp/test/GDPLogRead.xml

   [$[Get Code]]
   1. Run the model.
      bash-3.2$ $PTII/bin/vergil -capecode $PTII/ptolemy/actor/lib/jjs/modules/gdp/test/GDPLogRead.xml
      Updating local copy of the accessors repository.
      In "/Users/cxh/ptII/org/terraswarm/accessor/accessors", about to execute:
          svn update --non-interactive --trust-server-cert --accept postpone
      Updating '.':
      svn: E175002: Unable to connect to a repository at URL 'https://repo.eecs.berkeley.edu/svn/projects/terraswarm/accessors/trunk/accessors'
      svn: E175002: OPTIONS of 'https://repo.eecs.berkeley.edu/svn/projects/terraswarm/accessors/trunk/accessors': Could not resolve hostname `repo.eecs.berkeley.edu': Host not found
      (https://repo.eecs.berkeley.edu)
      All Done
      Could not update the accessors repository. Using local version.
      GDPLogRead: GDPLogRead.initialize() (Thread[GDPLogRead,1,main])
      gdp_api.c gdp_parse_name(GDPLogRead1,)
      gdp_api.c gdp_parse_name(): about to run sha256
      gdp_api.c gdp_parse_name(): 854d1f3f2888dbbee867792720fbf3589d328443712231bb958cde82ff6ca4f
      gdp_api.c: gdp_gcl_open():
      gdp_api.c: gdp_gcl_open(): EP_STAT_ISOK(): Setting *pgcl (0x7f9a383b32b0) to 0x7f9a382a4130!
      GDPLogRead: GDPLogRead.initialize() done (Thread[GDPLogRead,1,main])
      TestDisplay: foo (Thread[GDPLogRead,1,main])
      TestDisplay: data was null? (Thread[GDPLogRead,1,main])
      TestDisplay: data was null? (Thread[GDPLogRead,1,main])
      TestDisplay: data was null? (Thread[GDPLogRead,1,main])
      TestDisplay: data was null? (Thread[GDPLogRead,1,main])
      WARNING: TestSpontaneous: Model stopped before executing actions (e.g. producing outputs) scheduled for execution at times [6.0]
      2052 ms. Memory: 460288K Free: 292845K (64%)

      [$[Get Code]]
      Your output will vary, but the main thing is that the line
      TestDisplay: foo (Thread[GDPLogRead,1,main])

      [$[Get Code]]
      appears. This indicates that the Cape Code model was able to read the string foo from the log.
   2. The next step is to create a Composite Accessor and run it using Node:
      1. Double on the AccessorCodeGenerator
      2. Click on Generate. org/terraswarm/accessor/accessors/web/gdp/test/auto/GDPLogRead.js will be created and executed. The output will include the following:
         gdp_api.c: gdp_gcl_open(): EP_STAT_ISOK(): Setting *pgcl (0x104819ea0) to 0x103a03dc0!
         >>> recno 1, len 3, ts 2016-07-19T15:12:19.949130000Z
         00000000 66 6f 6f
         f o o
         

         [$[Get Code]]
         The above is generated by debugging in the C code. The output will also include:
         exiting with status OK [67896525 = 0x40c04cd]
         GDPLogRead.read() data:[object Object]
         GDPLogRead.read() done
         {"err":{"error_isok":true,"error_code":"0x40c04cd","error_msg":"exiting with status OK [67896525 = 0x40c04cd]"},"records":[{"recno":1,"timestamp":"2016-07-19T15:12:19.949130000Z","value":"foo"}]}

         [$[Get Code]]
         The above is from the TestDisplay accessor. Note that the value foo appears at the end of the line.

Invoking a Composite Accessor using just node

When the Generate button was depressed, something similar to the following output should have appeared:

1. To run the Composite Accessor outside of Cape Code using Node:
   bash-3.2$ cd $PTII/org/terraswarm/accessor/accessors/web/gdp/test/auto/
   bash-3.2$ ls -l GDPLogRead.js
   -rw-r--r--  1 cxh  staff  3156 Jul 19 08:52 GDPLogRead.js
   bash-3.2$ node ../../../{$HOSTS_NODE}/nodeHostInvoke.js -timeout 16000 gdp/test/auto/GDPLogRead
   Reading accessor at: /Users/cxh/ptII/org/terraswarm/accessor/accessors/web/gdp/test/auto/GDPLogRead.js
   
   ...
   
   GDPLogRead.read() data:[object Object]
   GDPLogRead.read() done
   {"err":{"error_isok":true,"error_code":"0x40c04cd","error_msg":"exiting with status OK [67896525 = 0x40c04cd]"},"records":[{"recno":1,"timestamp":"2016-07-19T15:12:19.949130000Z","value":"foo"}]}

   [$[Get Code]]

Troubleshooting

gdplogd and gdp_router

Use ps to check that the gdplogd and gdp_router daemons are working:

If they are not running, then rerun the gdp/test/setupAndRun.sh script

gdp-reader

Use the gdp-reader binary to read from the log, see above.

If you cannot open the GCL:

then perhaps the GCL was not yet created, see the gcl-create step above

How the Node/JavaScript/GDP interface works

When node is run on a Composite Accessor:

The following occurs

1. ../../../hosts/node/nodeHostInvoke.js refers to $PTII/org/terraswarm/accessor/accessors/web/hosts/node/nodeHostInvoke.js. That file reads GDPLogRead.js, which has
   exports.setup = function() {
   ...
       var GDPLogRead = this.instantiate('GDPLogRead', 'gdp/GDPLogRead.js');

   [$[Get Code]]
2. The accessor mechanism looks for the gdp/GDPLogRead.js 'accessor, which is found at accessors/web/gdp/GDPLogRead.js
3. GDPLogRead.js requires a module:
   var GDP = require('gdp');

   [$[Get Code]]
4. The gdp module is found at accessors/web/hosts/node/node_modules/@accessors-modules/gdp/. In that directory, package.json states that gdp.js is the main file:
     "main": "gdp.js",

   [$[Get Code]]
5. accessors/web/hosts/node/node_modules/@accessors-modules/gdp/gdp.js loads the module that was installed:
   var gdpjs = require('@terraswarm/gdp');

   [$[Get Code]]
6. $PTII/node_modules/@terraswarm/gdp/package.json is loaded, and package.json states that index.jss is the main file:
     "main": "index.js",

   [$[Get Code]]
7. $PTII/node_modules/@terraswarm/gdp/index.js ends up requiring ./gdpjs/gdpjs.js, which is $PTII/node_modules/@terraswarm/gdp/gdpjs/gdpjs.js.
8. $PTII/node_modules/@terraswarm/gdp/gdpjs/gdpjs.js defines a C-interface to the gdp

How to make a GDP function available in JavaScript

For testing purposes, it is easiest to edit the files in the $PTII/node_modules/@terraswarm/gdp/ directory, then update the corresponding files in $PTII/vendors/gdp/gdp/lang/js.

$PTII/node_modules/@terrswarm/gdp/gdpjs/gdpjs.js contains the interface between the C-based GDP API and JavaScript. In that file, there are interfaces to three shared libraries.

• libep - Eric Allman's Enhanced Portability Library
• libgdp - The Global Data Plane
• libgdpjs - A few small functions that make it easier to access parts of libgdp that are written using C Macros

To access a C-based GDP API function from JavaScript, look for the method signature in gdpjs.js. If the function is not there, then look for it in gdp/gdp.h and add it to gdpjs.js. After the call to ffi.Library, export the function:

For example, if we want to access the GDP ep_dbg_set() function, then in gdpjs.js:

The next step is to add the function to $PTII/node_modules/@terraswarm/gdp/index.js. In that file, gdpjs.js is required and then reexported:

Then, we update the gdp module to include the call. For the node host, the gdp module is defined in accessors/web/hosts/node/node_modules/@accessors-modules/gdp/gdp.js, which appears in the Ptolemy tree as $PTII/org/terraswarm/accessor/accessors/web/hosts/node/node_modules/@accessors-modules/gdp/gdp.js:

TroubleShooting

Cannot Read Property

Problem:

Solution: Make sure to instantiate the GDP object

Not

See Also

• GDP JavaScript Interface (swarmos wiki)
• GDP JS Callbacks
• GDP JS Notes - older notes about the GDP/JavaScript interface
• GDP - Using the GDP with Accessors 'Start here