Recent Changes - Search:

edit SideBar

C /

CModulesInNode

Summary

Accessors will need to bind interactions with novel technologies -- both software and hardware, which are inaccessible from JavaScript. Cape Code provides Nashorn to provide a JS bindings for Java methods. Node allows developers to author "add-ons" in C++, which can be accessed from Node using `require` keyword.

Node is an amalgam of:

  • libuv -- a cross-platform C library for asynchronous I/O that governs Node's event loop & provides threading operations
  • Node libraries -- a set of C++ libraries for built-in Node functions (including Buffers, sockets, etc.)
  • V8 -- Google's C++ implementation of ECMAScript, which provides javascript bindings for the above

The problem with Node is that all three of the above are moving targets, developed by different teams. That is, the "RTFM" mantra is particularly difficult to put into practice, as you need to read 3 manuals, with at least 3 of these frequently lying due to API changes. This page is set up to collate various bits of practical advice for anyone wishing to bind their C/C++ libraries into Node modules to build accessors.

Audience

This document assumes familiarity with C, C++, and threads.

TODO: outline C++11 features used in code examples as I write the code examples...

Documentation Resources

Addressing Documentation Shortcomings:

Code Snippets for Common Tasks

To build anything, you will need `node-gyp` installed globally:

sudo npm install -g node-gyp
[$[Get Code]]

For more information on how to use it, see the node-gyp section below.

Addon Basics

  1. Write a C++ file
  2. Create a bindings.gyp file (in essence, a Makefile)
  3. run node-gyp rebuild or node-gyp configure && node-gyp build to build your add-on
  4. run node and test: myAddon = require('./build/Release/myaddon');

Storing Persistent Variables

C++ functions that are accessed from a Node module will have the following signature:

void myFunction( const v8::FunctionCallbackInfo<Value>& args )
[$[Get Code]]

The args variable can be used to:

1. args.GetIsolate() to get a pointer to the javascript VM (for doing anything useful); 2. args.Length() to inquire as to the number of arguments 3. args[0]->IntegerValue() and alike functions to get to the data from JavaScript variables.

Note that when getting values from javascript, V8 will pass a v8::Local<T> handle to a Javascript datatype. v8::Local handles will be garbage collected as soon as they will exit the scope, "even if you are attempting to store them in some global variable". This is not a problem for data, as you usually will cast it to a C++ datatype for storage, but storing a Javascript callback and calling it from some other function requires to use v8::Persistent<T> handle.

To store a javascript callback:

v8::Persistent<v8::Function> storedJsCallback; // for clarity, as a global variable
...
void functionToStoreCallback( const v8::FunctionCallbackInfo<Value>& args )
{
        auto isolate = args.GetIsolate(); // get a pointer to the V8 javascript VM
        v8::Local<v8::Function> jsCallback = v8::Local<v8::Function>::Cast( args[0] ); // get a Local handle to a callback passed to your function
        storedJsCallback.Reset( isolate, jsCallback ); // due to breaking changes this call is poorly documented
}
[$[Get Code]]

Note that as of V8 v0.12, v8::Persistent<T>::New and v8::Persistent<T>::operator= are private. At the time of writing, you set a persistent handle by calling .Reset() and passing a local handle as a constructor argument: storedJsCallback.Reset( isolate, jsCallback ); (see above)

To call a stored Persistent<Function>, you will first need to convert it to a Local<Function>, as Persistent<Function> is not callable:

//Persistent<Function> storedJsCallback is a global as above
void functionToCallTheCallback( const FunctionCallbackInfo<Value>& args )
{
        auto isolate = args.GetIsolate();
        Local<Function> callable = Local<Function>::New( isolate, storedJsCallback ); // obtain a local handle
        // generate an array of javascript values to be passed as function arguments
        callable->Call(isolate->GetCurrentContext(), ...argc..., ...argv...);
        // for example:
        // callable->Call(isolate->GetCurrentContext(), args.Length(), args );
        // will call the stored callback with the arguments passed to this function
}
[$[Get Code]]

Async Operation

Asynchronous operations with Node require interacting with libuv. The primary guideline for Node extensions is that the only thread that can access javascript is the Node event loop. The TLDR of this section that points to the necessary documentation is:

spawning an async operation:

  • uv_work_t
  • uv_queue_work()

waking up the Node event loop (V8 / javascript) from a different thread:

* uv_async_t
* uv_async_send()

You own the new thread:

That is, you would like to spawn a new thread that will do some work. After completion, you will need the main event loop in Node to call a javascript function with the results of the computation as arguments. This requires some setup:

  1. define the function that is called from javascript that prepares the async operation
# convert javascript arguments to C++ datatypes
# store these C++ datatypes in a structure that will be accessible to the worker thread
  1. define the function that will be called from a different thread that does the work
# this function is a plain vanilla C++ function that has no access to the V8 Isolate, or any JavaScript variables
  1. define the third function that will be called from the event thread when the operation completes
# this function will be called from the event loop thread, and therefore can interact with V8/Javascript
# use Isolate::GetCurrent() to obtain a pointer to the V8 instance for creating javascript variables & calling functions

Note that because we are dealing with Javascript callbacks, we will be using Persistent<Function> handle. If the mechanism seems confusing, review Storing Persistent Variables above.

#include <node.h>
#include <uv.h>
#include <iostream>
#include <string>
#include <thread>

using namespace v8;
using namespace std;

// the structure for data accessible from different threads:
struct Baton
{
    // the uv_work_t must be the initial element to ensure safe casts of uv_work_t* to Baton*
    uv_work_t request;
    Persistent<Function> callback; // callback if you need data back in JS
    int countdown; // whatever else data fields as necessary
};

void AsyncTestWork (uv_work_t* req);
void AsyncTestAfter(uv_work_t* req, int status);
void AsyncTestPrep(const FunctionCallbackInfo< Value >& args) // this function is called from Node
{
    cout << "AsyncTestPrep called from thread " << std::this_thread::get_id() << '\n';
    auto isolate = args.GetIsolate();

    Baton* baton = new Baton();
    baton->request.data = baton; // set the uv_work_t data field to point to our Baton
    baton->callback.Reset(isolate, Local<Function>::Cast(args[0]));
    baton->countdown = 3;

    uv_queue_work(uv_default_loop(), &baton->request, AsyncTestWork, AsyncTestAfter);
    args.GetReturnValue().Set(Undefined(isolate));
}


void AsyncTestWork(uv_work_t* req)
{
    // This method will run in a separate thread where you can do your blocking background work.
    // In this function, you cannot under any circumstances access any V8/node js variables. (Use the Baton)
    cout << "AsyncTestWork Enter\n";
    cout << "AsyncTestWork called from thread " << std::this_thread::get_id() << '\n';
    Baton* baton = static_cast<Baton*>(req->data); // data field of req points to the Baton as per above
    baton->countdown -= 1;  // async subtraction -- the work we're doing
    cout << "AsyncTestWork Exit\n";
}

void AsyncTestAfter(uv_work_t* req, int status)
{
    // This is what is called after the 'task' is done, you can now move any data from
    // Baton to the V8/Nodejs space and invoke callback functions
    cout << "AsyncTestAfter Enter\n";
    cout << "AsyncTestAfter called from thread " << std::this_thread::get_id() << '\n';

    Baton* baton = static_cast<Baton*>(req->data);
    cout << "baton countdown is : " << baton->countdown << '\n';

    auto isolate = Isolate::GetCurrent(); // get the JS VM
    HandleScope scope(isolate);

    Local<Value> argv1[] = { Number::New( isolate, baton->countdown) };
    cout << "created javascript arguments from countdown\n";

    // Call back to the JS function(s):
    auto callbackHandle = Local< Function >::New( isolate, baton->callback );
    callbackHandle->Call( isolate->GetCurrentContext()->Global(), 1, argv1);

    // resubmit if we're not done, otherwise cleanup:
    if (baton->countdown > 0) {
        uv_queue_work(uv_default_loop(), &baton->request, AsyncTestWork, AsyncTestAfter);
    } else {
        baton->callback.Reset();
        delete baton;
    }
}

void Init(Local< Object > exports)
{
    NODE_SET_METHOD( exports, "asyncTest", AsyncTestPrep );
}

NODE_MODULE( addon, Init )
[$[Get Code]]

The `bindings.gyp` file for creating the add-on:

{
  "targets": [
    {
      "target_name": "async",
      "sources": [ "threadworker.cpp" ],
      "cflags": ["-Wall", "-std=c++11"],
      'xcode_settings': {
        'OTHER_CFLAGS': [
          '-std=c++11'
        ],
      },
      'msvs_settings': {
        'VCCLCompilerTool': {
          'ExceptionHandling': 1 # /EHsc
        }
      },
      'configurations': {
        'Release': {
          'msvs_settings': {
            'VCCLCompilerTool': {
            'ExceptionHandling': 1,
          }
        }
      }
      },
      "conditions": [
        [ 'OS=="mac"', {
            "xcode_settings": {
                'OTHER_CPLUSPLUSFLAGS' : ['-std=c++11','-stdlib=libc++'],
                'OTHER_LDFLAGS': ['-stdlib=libc++'],
                'MACOSX_DEPLOYMENT_TARGET': '10.7' }
            }
        ]
      ]
    }
  ]
}
[$[Get Code]]

Sample output:

> require('./build/Release/async').asyncTest(function(v){ console.log("hello from javascript : " + v); });
AsyncTestPrep called from thread 0x7fff7b9e6000
AsyncTestWork Enter
AsyncTestWork called from thread 0x70000221b000
AsyncTestWork Exit
undefined
> AsyncTestAfter Enter
AsyncTestAfter called from thread 0x7fff7b9e6000
baton countdown is : 2
created javascript arguments from countdown
hello from javascript : 2
AsyncTestWork Enter
AsyncTestWork called from thread 0x700001215000
AsyncTestWork Exit
AsyncTestAfter Enter
AsyncTestAfter called from thread 0x7fff7b9e6000
baton countdown is : 1
created javascript arguments from countdown
hello from javascript : 1
AsyncTestWork Enter
AsyncTestWork called from thread 0x700001a18000
AsyncTestWork Exit
AsyncTestAfter Enter
AsyncTestAfter called from thread 0x7fff7b9e6000
baton countdown is : 0
created javascript arguments from countdown
hello from javascript : 0
[$[Get Code]]

Notice that although AsyncTestWork is called in various threads, both AsyncTestPrep and AsyncTestAfter are always called from the main event loop thread (0x7fff7b9e6000 in the sample output above).

Hardware owns the new thread:

Node-GYP Notes

If you were thinking "I love learning new build systems because I love it when I have to learn entirely unfamiliar ways to pass some flags to my compiler" boy do I have good news for you! Node uses GYP (Generate Your Projects) meta build system, which Google wrote out of spite for CMake. (This sounds like a joke, but I assure you it isn't.)

To build a node addon, you need to write a bindings.gyp makefile.

TODO: post my own that build on Mac & Linux and write something snarky about Windows.

(TODO: check if Microsoft is our sponsor before proceeding with the last part...)

For now, please see "binding.gyp" files out in the wild for examples.

See Also
Edit - History - Print - Recent Changes - Search
Page last modified on October 01, 2016, at 07:59 pm