How to create a webhook with Node.js

How to Build a webhook from scratch

In order to build a webhook from scratch, an HTTPS server that is running on a machine open to the public internet with a certificate from a recognized authority is required. The server code is a very basic HTTPS server that can receive POST requests like the ones that TradeLens uses to send subscription messages. The server code can be found here: https://github.com/ibmgtd/sip-sample-webhook-node

The Infrastructure Required

To create the webhook infrastructure is required to support it:

1. The webhook has to run on a machine that's accessible via the public Internet. We can create an IBM Cloud virtual machine for that purpose, using your account. 
2. A valid security certificate is required and you can get one from Let's Encrypt. (The certificate will have to be renewed every 90 days, which is easy to do.)
3. Let's Encrypt also requires that the host machine be accessible via a DNS name (something like foo.bar.com), not just via a numeric IP address. You have to create a DNS domain and add your server to that domain. 
4. Once you have these three things, you can run the attached server.js on your public machine, and it should be able to receive subscription messages from TradeLens. 
5. The setup in Apache and node.js is just to accept incoming subscription messages and display them on the console. To make this useful, you'll also want to add a database to have persistent storage for the subscription messages. You could use an IBM Cloud database for that, or we could install Postgres or another database on the server machine; then modify the server code to write to that database.
6. Finally, you should also consider how your users will pull data out. Tools like Power BI or Tablo, for example.

Testing the webhook

You can test the node.js server on your local machine. You need to install npm and node.js; also curl and openssl. Once you have those things, you can exercise the server as follows, and as shown in the  recording found here :

Step 1. Store the two attached .pem files on your local machine. Those two files form the self-signed certificate, in the name of /C=US/ST=Pennsylvania/L=Erie/O=MyCo/OU=PDH/CN=Test2 . Modify the two fs.readFileSync() lines at the top of server.js to specify the full path names to the .pem files. 

Step 2. Run the server in one window, via this command:   node server.js      . You may have to do the following first to install a few optional packages:

npm install body-parser
npm install express
npm install lodash

When successful, the server will be listening for HTTP requests on port 8080, and will listen for HTTPS requests on port 8443.

Step 3. Confirm that the server is listening and that the certificate works, via this command in a separate window:

openssl s_client -connect localhost:8443

You should get a page or two of output, with the crucial lines being these (showing the self-signed certificate issuer and subject):

Certificate chain
 0 s:/C=US/ST=Pennsylvania/L=Erie/O=MyCo/OU=PDH/CN=Test2
   i:/C=US/ST=Pennsylvania/L=Erie/O=MyCo/OU=PDH/CN=Test2

Hit control-C to break out of the openssl connection.

Step 4. Still in the second window, send a couple of JSON messages to the server via curl as follows (two long lines):

curl --insecure -s -X POST --header 'Content-type: application/json' -d '[{ "firstField": "Test message from TradeLens Support", "sender": "XXXXXXXXXXXXX" }]' https://localhost:8443

curl --insecure -s -X POST --header 'Content-type: application/json' -d '[{ "firstField": "Another message from TradeLens Support", "sender": "Same as the first one: XXXXXXXXXXXX" }]' https://localhost:8443

This emulates TradeLens sending a JSON subscription message. You should see "Message received" in the second window, and you should see the JSON in the server window.

Step 5. The server stores the messages it receives in an in-memory array. To see the contents of that array, do this in the second window:

curl --insecure -s -X GET https://localhost:8443

As noted above, the server would save the messages in a database, and you'd retrieve them via some other tool. The curl command here just shows that the basic server did receive the messages and still has them in memory.

Note:  "--insecure" on the curl commands is due to a self-signed certificate and tells curl that we are OK with that insecurity.