neuropil API

Note

The neuropil API makes use of types defined in the standard C headers stdint.h, stdbool.h and stddef.h.

neuropil.h

typedef void np_context

An opaque object that denotes a neuropil application context.

np_context *np_new_context(struct np_settings *settings)

Creates a new neuropil application context.

Parameters
  • settings – a np_settings structure used to configure the application context.

Returns

a pointer to the newly created application context.

typedef void np_subject

An blake2b obfuscated message subject to transport data

void np_default_settings(struct np_settings *settings)

Initializes a np_settings structure to the default settings.

Parameters
  • settings – a pointer to the np_settings structure to be initialized.

struct np_settings

The np_settings structure holds various run-time preferences available to neuropil.

uint32_t n_threads

Controls the maximum number of threads used by neuropil at any given time. The default is 3.

char log_file[256]

Pathname of a file that neuropil will log to. The default is a "<timestamp>_neuropil.log" where timestamp is a decimal millisecond UNIX timestamp.

uint8_t leafset_size

specifies the size of the leafset table (DHT entries near to our own hash value)

uint16_t jobqueue_size

The size of the internally used jobqueue. The default is 512 entries, depending on the number of threads this should be sufficient for many use cases. High throuput cloud nodes could need larger jobqueues.

struct np_token *np_new_identity(np_context *ac, double expires_at, uint8_t secret_key[NP_SECRET_KEY_BYTES])

Creates a new neuropil identity.

Parameters
  • ac – a neuropil application context.

  • expires_at – expiry date of the identity in seconds since the Unix epoch.

  • secret_key – the secret key used by the identity. If NULL is supplied a random key is generated.

Returns

an identity token.

enum np_return np_set_identity(np_context *ac, struct np_token identity)

Sets the identity used by the neuropil node.

Parameters
  • ac – a neuropil application context.

  • identity – the identity to use.

Returns

np_ok on success.

Status

Meaning

np_invalid_argument

Identity is no longer valid, or uses an insecure key.

np_invalid_operation

The application is in use (can not change the identity at run-time.)

enum np_return np_listen(np_context *ac, const char *protocol, const char *host, uint16_t port)

Binds a neuropil application context to a listening address.

Parameters
  • ac – a neuropil application context.

  • protocol – a string denoting the underlying protocol to be used. Currently, only “udp4” is supported.

  • host – the hostname to listen on. I.e., “localhost” to listen on the loopback interface.

  • port – the port to listen on. If port is zero, the default port 3141 is used.

Returns

np_ok on success.

Status

Meaning

np_invalid_argument

Host could not be resolved.

np_invalid_operation

No identity is set for the application context.

enum np_return np_get_address(np_context *ac, char *address, uint32_t max)

Gets the absolute address of the neuropil node within the overlay network.

Parameters
  • ac – a neuropil application context.

  • address – a pointer to the address string to be written to.

  • max – the size in bytes of address. Should be large enough to hold the resulting address string. The required space depends on the node’s host name (i.e., 1000 bytes should be more than enough for most uses.)

Returns

np_ok on success.

Status

Meaning

np_invalid_argument

Address is not large enough to hold the address string.

np_invalid_operation

No listening address is bound for the application context. (Call np_listen() first.)

enum np_return np_join(np_context *ac, const char *address)

Adds a bootstrap node to be used by this node to join the neuropil network.

Parameters
  • ac – a neuropil application context.

  • address – a string that denotes an absolute address as obtained by np_get_address().

Returns

np_ok on success.

Status

Meaning

np_invalid_argument

Address is malformed.

enum np_return np_generate_subject(np_subject subject, const char *text, size_t length)

Creates the binary representation of a message subject. This is an re-entrant version, using the same subject field will use the existing np_subject as a seed for the text.

Parameters
  • subject – the final np_subject field that can be used in later calls to the library

  • text – the text that the subject should be based on

  • length – the length of text in bytes.

Returns

np_ok on success.

Status

Meaning

np_invalid_argument

Length exceeds the maximum message size supported by this implementation.

enum np_return np_send(np_context *ac, np_subject subject, const uint8_t *message, size_t length)

Sends a message on a given subject.

Parameters
  • ac – a neuropil application context.

  • subject – the subject to send on.

  • message – a pointer to a buffer containing the message to be sent. The message could be, for instance, encoded using MessagePack.

  • length – the length of message in bytes.

Returns

np_ok on success.

Status

Meaning

np_invalid_argument

Length exceeds the maximum message size supported by this implementation.

enum np_return np_add_receive_cb(np_context *ac, np_subject subject, np_receive_callback callback)

Adds a callback to be executed when receiving a message on a given subject. It is possible to add more than one receive callback for a given subject, in which case they are run in the order in which they were added.

Parameters
  • ac – a neuropil application context.

  • subject – the subject to receive on.

  • callback – a pointer to a function of type np_receive_callback that denotes the callback to be added.

Returns

np_ok on success.

Status

Meaning

np_invalid_operation

The maximum number of receive callbacks supported by the implementation for the given subject is exceeded.

bool np_receive_callback(struct np_message *message)

Receive callback function type to be implemented by neuropil applications. A message receipt is considered to be acknowledged if all receive callbacks associated with the subject returned (true). Once a receive callback returns (false), the message is considered rejected and no further callbacks for the subject are executed.

Parameters
  • message – a pointer to a np_message structure.

Returns

a boolean that indicates if the receipt was acknowledged (true) or rejected (false.)

struct np_message

Structure that holds a received message and some metadata about that message.

uint8_t *data

A pointer to a buffer that contains the received message.

size_t data_length

The length of data in bytes.

char uuid[NP_UUID_BYTES]

A universally unique identifier for the message.

np_id from

The identity fingerprint of the message.

np_id subject

The fingerprint of the message subject.

double received_at

Unix timestamp that denotes the time the message was received.

enum np_return np_set_mx_properties(np_context *ac, const char *subject, struct np_mx_properties properties)

Configure message exchange semantics for a given subject. The default is best-effort message delivery without any attempt at retransmission and if delivered messages are guaranteed to be delivered once only.

Parameters
  • ac – a neuropil application context.

  • subject – the subject to configure message exchange semantics on.

  • properties – a pointer to a np_mx_properties structure that describes the semantics to be applied.

Returns

np_ok on success.

Status

Meaning

np_invalid_argument

The properties structure is invalid.

struct np_mx_properties

Structure that defines message the exchange semantics for a given subject.

np_mx_ackmode ackmode

Acknowledgement strategy used in message exchange. The default is NP_MX_ACK_NONE (i.e., fire and forget.)

np_mx_role role

role of the node for this data transfer. The default is not set. When calling np_send the library creates the role of NP_MX_PROVIDER, if calling np_add_receive_cb the role NP_MX_CONSUMER is used

np_mx_audience_type audience_type

the intended audience for this data transfer. see np_mx_audience_type for the specific meaning of each. The default is NP_MX_AUD_PUBLIC

np_mx_cache_policy cache_policy

Cache policy used for queuing inbound messages. The default is NP_MX_FIFO_REJECT (i.e., messages are delivered first in, first out, and messages that would overflow the queue will be rejected.)

uint8_t max_parallel

The maximum number of outbound messages that may be in-flight at a given moment. The default is one.

uint8_t max_retry

The maximum number of times a message will be resent after it has been rejected. The default is zero.

double intent_ttl
double intent_update_after

The duration of validity of issued message intents and the duration after which message intents are to be refreshed in seconds. Intent_update_after should always be less than intent_ttl. This setting impacts the fail-over latency between receivers leaving and joining the network. Senders will remain unaware of new receivers for up to intent_update_after seconds.

The default for intent_ttl is 30 seconds. The default for intent_update_after is 20 seconds

double message_ttl

Maximum duration for individual message delivery in seconds. The default is 20 seconds.

bool once_only

Boolean that indicates if messages should be ensured to be delivered once only. If this is false duplicate messages may be delivered to the application. The default is true.

enum np_mx_cache_policy

Mode

Description

NP_MX_FIFO_REJECT

Messages are delivered in FIFO order, excessive messages are rejected.

NP_MX_FIFO_PURGE

Messages are delivered in FIFO order, excessive messages are silently discarded.

NP_MX_LIFO_REJECT

Messages are delivered in LIFO order, excessive messages are rejected.

NP_MX_LIFO_PURGE

Messages are delivered in LIFO order, excessive messages are silently discarded.

enum np_mx_ackmode

Mode

Description

NP_MX_ACK_NONE

Message transmissions need not be acknowledged to be considered successful.

NP_MX_ACK_DESTINATION

Message transmissions need to be acknowledged by destination node to be considered successful.

NP_MX_ACK_CLIENT

Message transmissions need to be acknowledged by a receive callback to be considered successful.

enum np_mx_role

Mode

Description

NP_MX_PROVIDER

node is the sender of messages

NP_MX_CONSUMER

node is the receiver of messages

NP_MX_PROSUMER

node will send an receiver messages on this subject

enum np_mx_audience_type

Mode

Description

NP_MX_AUD_PUBLIC

public data channel, everybody can subscribe to

NP_MX_AUD_VIRTUAL

virtual data channel, only token will be exchanged

NP_MX_AUD_PROTECTED

protected data channel, audience_id identifies the mutual peer

NP_MX_AUD_PRIVATE

private data channel, subject obfuscated with np_generate_subject

enum np_return np_set_authorize_cb(np_context *ac, np_aaa_callback callback)

Sets the authorization callback used to control access to message exchanges. The provided callback is responsible for judging whether the identity that issued a given token is permitted to exchange messages over a given subject. If no authorization callback is set all message exchanges will be rejected.

Parameters
  • ac – a neuropil application context.

  • callback – a pointer to a function of type np_aaa_callback that denotes the callback to be set.

Returns

np_ok on success.

Status

Meaning

np_invalid_operation

An authorization callback has already been set for this application context.

enum np_return np_set_authenticate_cb(np_context *ac, np_aaa_callback callback)

Sets an additional authentication callback used to authenticate nodes. Such a callback can be used to extend the authentication provided by neuropil to further validate token based on application extensions. If no such callback is set only standard neuropil authentication is performed. Note that authenticated nodes are permitted to join the overlay network.

Parameters
  • ac – a neuropil application context.

  • callback – a pointer to a function of type np_aaa_callback that denotes the callback to be set.

Returns

np_ok on success.

Status

Meaning

np_invalid_operation

An additional authentication callback has already been set for this application context.

bool np_aaa_callback(struct np_token *aaa_token)

AAA callback function type to be implemented by neuropil applications. These functions are to inspect and verify the contents of the aaa_token they are provided and either accept or reject the token.

Parameters
  • aaa_token – a pointer to a np_token structure to be verified.

Returns

a boolean that indicates if the token was accepted (true) or rejected (false.)

struct np_token

A record used for authentication, authorization and accounting purposes. When np_token records are transmitted or received over the network they are accompanied by a cryptographic signature that must match the token’s public_key field. A token received through the neuropil API are guaranteed to be authentic: i.e., their integrity is validated. Applications are responsible to verify the issuer of a given token as denoted by the public_key.

np_id realm

Optionally, token can specify a third party authority that governs the validity of token in a realm.

np_id issuer

The fingerprint of the identity that issued the token.

char subject[255]

A subject that denotes the token’s purpose.

np_id audience

A fingerprint that denotes the intended audience of the token.

double issued_at
double not_before
double expires_at

Timestamps encoded as Unix time in seconds that denote issue date and validity duration of the token. These validity periods are validated by neuropil.

uint8_t extensions[]
size_t extension_length

A buffer of extension data of extension_length bytes represented as a MessagePack encoded map.

uint8_t public_key[NP_PUBLIC_KEY_BYTES]
uint8_t secret_key[NP_SECRET_KEY_BYTES]

The key pair associated with the token. Foreign token have the secret_key unset (all zero).

void np_get_id(np_id (*id), char *string, size_t length)

Computes the fingerprint (or overlay address) of a serialized object.

Parameters
  • id – a np_id to be written.

  • string – the data to be hashed.

  • length – the length of the input data in bytes. If length is zero, string is expected to be a zero-terminated string.

np_get_id() constructs fingerprints by means of a cryptographic, one-way hash function. Hence, fingerprints are unique, unforgeable object identities. Given an object, any party can compute its unique fingerprint, but no party is able to forge an object that hashes to a particular fingerprint.

typedef uint8_t np_id[NP_FINGERPRINT_BYTES]

The type np_id denotes both a fingerprint and a virtual address in the overlay network implemented by neuropil. It is represented as a consecutive array of NP_FINGERPRINT_BYTES bytes.

enum np_return np_run(np_context *ac, double duration)

Runs the neuropil event loop for a given application context for a specified duration*. During the execution of the event loop incoming and outgoing messages are transmitted and received, and the associated callbacks are executed.

Parameters
  • ac – a neuropil application context.

  • duration – the duration in seconds allotted to execute the event loop. If duration is zero np_run() will return as soon as it has processed all outstanding events.

Returns

np_ok on success.

enum np_return

This type denotes the set of status codes returned by various functions in the neuropil API. Possible values include:

Status

Meaning

np_ok

The operation was successful (no error occurred.)

np_invalid_argument

The operation failed because of an invalid argument.

np_invalid_operation

The operation is not permitted at this time.

In order to accurately interpret error codes refer to the documentation of the specific function in question.

size_t NP_SECRET_KEY_BYTES
size_t NP_PUBLIC_KEY_BYTES

Constants that denote the lengths in bytes of the private and public key parts used by neuropil, as found in np_token.

size_t NP_FINGERPRINT_BYTES

Constant that denotes length in bytes of both fingerprints and virtual addresses in the overlay network implemented by neuropil. Specifically, this is the size of np_id.

size_t NP_UUID_BYTES

Constant that denotes the length in bytes of message UUID_s.

neuropil_data.h

@brief Initialises the provided memory allocation as datablock

@param block The memory block provided by the user (malloc’ed ) @param block_length The memory block size @return enum np_data_return

@brief merges src datablock into dest datablock. overwrites existing keys

typedef void np_datablock_t

An opaque object that denotes a neuropil key/value store.

enum np_data_return np_init_datablock(np_datablock_t *block, size_t block_length)

Creates a new neuropil key value store.

Parameters
  • block – the memoryblock which should be initialized to contain a np_datablock_t

  • block_length – a size_t the size of the block input parameter

Returns

np_ok on success

struct np_data_conf

The configuration of a single key in an np_datablock_t

enum np_data_type

This type denotes the set of data types relevant for np_datablock_t key/value elements Possible values include:

Status

Meaning

NP_DATA_TYPE_BIN

Define the key to hold binary data in its value container

NP_DATA_TYPE_INT

Define the key to hold an integer in its value container

NP_DATA_TYPE_STR

Define the key to hold a string in its value container (Uses ASCII encoding)

typedef bool np_data_iterate_cb(struct np_data_conf *out_data_config, np_data_value *out_data, void *userdata)

Callback definition for the :c:function:`np_iterate_data` function

enum np_data_return np_iterate_data(np_datablock_t *block, np_data_iterate_cb callback, void *userdata)

Iterates over every np_datablock_t key/value elements

neuropil_attributes.h

The purpose of this API is to enable the user to create and use arbitrary attributes on np_token and message intent level.

Attributes can then be e.g. used to:

  • publish searchable metadata (e.g. the location of a temperature sensor or the cost of a data sample)

  • create a SessionID based authentication. (see: NP_ATTR_IDENTITY_AND_USER_MSG)

  • many more …

enum np_msg_attr_type

This type denotes the set of scope levels relevant for a given attribute.

Possible values include:

Value

Meaning

NP_ATTR_USER_MSG

The attribute will be available in every USER message. (end-to-end encrypted)

NP_ATTR_IDENTITY

The attribute will be appended to the identity np_token and will then be transfered to every node receiving a _NP.JOIN.REQUEST request. (end-to-end encrypted)

NP_ATTR_INTENT

The attribute will be appended to every _NP.MESSAGE.DISCOVER.RECEIVER and _NP.MESSAGE.DISCOVER.SENDER message. (transport encrypted)

NP_ATTR_IDENTITY_AND_USER_MSG

combine NP_ATTR_USER_MSG and NP_ATTR_IDENTITY

NP_ATTR_INTENT_AND_USER_MSG

combine NP_ATTR_USER_MSG and NP_ATTR_INTENT

NP_ATTR_INTENT_AND_IDENTITY

combine NP_ATTR_INTENT and NP_ATTR_IDENTITY

enum np_return np_set_ident_attr_bin(struct np_token *ident, enum np_msg_attr_type inheritance, char key[255], unsigned char *bin, size_t bin_length)

Sets an identity level wide attribute

Parameters
  • ident – the np_token to set the attribute for

  • inheritance – the np_msg_attr_type inheritance level for this attribute

  • key – the network wide identifier for this attribute

  • bin – the attribute value as binary blob

  • bin_length – the length of the blob

Returns

np_ok on success

enum np_return np_set_mxp_attr_bin(struct np_mx_properties *prop, enum np_msg_attr_type inheritance, char key[255], unsigned char *bin, size_t bin_length)

Sets an message exchange level wide attribute

Parameters
  • prop – the np_mx_properties to set the attribute for

  • inheritance – the np_msg_attr_type inheritance level for this attribute, only non NP_ATTR_IDENTITY… values are respected.

  • key – the network wide identifier for this attribute

  • bin – the attribute value as binary blob

  • bin_length – the length of the blob

Returns

np_ok on success