C CL Porting Guide Draft

Table of Contents

1.Preface

2.Prerequisites

3.Overview

4.Build time options

Shared options

5.Mandatory Porting API

iotcs_port_system.h:

iotcs_port_queue.h:

iotcs_port_crypto.h:

iotcs_port_ssl.h:

6.Optional Porting API

Dynamic Memory Allocation API

iotcs_port_memory.h:

Optional thread related API

iotcs_port_thread.h:

iotcs_port_mutex.h

Optional diagnostic capability porting API

iotcs_port_diagnostics.h:

Optional for Message Dispatcher

iotcs_port_queue.h:

Optional for Long Polling support

iotcs_port_ssl.h:

Optional for automatic device time adjustment

iotcs_port_system.h:

7.Porting MQTT support

Paho Embedded porting layer

IoT Library Paho bindings

Paho Embedded port example

8.UTAM porting API

iotcs_port_tam.h:

Default TAM support methods (Obsolete and Deprecated)

iotcs_port_tam.h:

9.Custom TAM

10.Examples of porting

11.Build system adjustment

How to add custom porting layer

How to add new compiler

How to add custom TAM

Custom TAM:

Custom port TAM:



1. Preface

This guide is intended for people who would like to port C CL to a new platform that is not currently supported out of the box.

  • 2. Prerequisites

    Reader must be familiar with:

    1. IoT Architecture

    2. IoT Client

    3. Device Model

    4. TAM

    5. Unified TAM

    3. Overview

    The C Client Library (CL) architecture was designed with porting in mind which makes porting to a new platform a rather simple process.

    Here is a diagram of current design: Scheme of the C Client Library architecture

    There are three modules: shared part, porting layer and trusted asset manager (TAM ). Each module has its own API. Public API for shared part, Porting API for Porting Layer and TAM API for TAM. Let's consider each part.

    Shared part is written in C99 language using standard C99 library calls which makes it portable across platforms that can use the C99 compiler. The Shared part API is a public API that can be found in binary form in the./iotcs/csl/posix/include directory.

    However, the C99 standard library does not contain some functionality required to implement the C CL. All these calls are encapsulated in the Porting API which is implemented by porting layer. Basically the Porting API consists of some API subsets: thread API, mutex API, queue API, SSL/TLS connection etc. Some of these APIs are used by the Shared part (mandatory porting API), while others depend on a particular C CL configuration, and may or may not be required (optional porting API). The Porting API can be found in a source bundle in the directory ./iotcs/csl/posix/src/include/port (this path is for the Posix release, for other releases please substitute posix with the release platform name.)

    The last is a TAM API that can be found in a source bundle in directory ./iotcs/csl/posix/src/shared/trusted_assets_manager/iotcs_tam.h (again, this is the path for Posix release.) The Trusted Assets Manager API defines methods for handling trust material used for activation and authentication to the IoT Cloud Service. It is responsible for storing the server URL and port, server certificate, activation id, endpoint id, RSA public and private key bound to endpoint id. Besides storing, the TAM API also provides methods for encryption using a shared secret and a private key. Depending on the capability of the device as well as on the security requirements implementations of this interface, you may store sensitive trust material in a plain persistent store file system, or in a hardware keystore, such as a pluggable authentication module (PAM), or in a secure token.

    The C CL provides its own TAM API implementation called the Unified TAM (for more info please see for more info please see Oracle Internet of Things cloud provisioning information). This TAM stores all the information in an encoded form on a local file system in a well-defined format compatible across different IoT Client implementations (for example, Java SE IoT CL or JS IoT CL.) Implementation of Unified TAM has a dedicated porting layer called: the UTAM porting layer. So porting UTAM means to implement the UTAM porting layer. The UTAM porting API is declared in ./iotcs/csl/posix/src/include/port/iotcs_port_tam.h. If a target platform has hardware support for cryptography then this hardware can be utilized in UTAM porting layer implementation to increase library performance.

    UTAM also supports an obsolete Default TAM format. This format also stores all the information on a local file system but the information isn't encoded - the file integrity is protected by a password and the file format is specific for the C CL. This format is deprecated and only supported for compatibility reasons. It could be removed in the future so please don't use it unless absolutely necessary.



    There may be situations where the Unified TAM is inappropriate:

    In these situations you can create your own custom TAM implementation instead of using UTAM. However, your custom TAM must provide TAM API to C CL library.

    So minimal porting process would be:

    1. Implement mandatory part of Porting API.

    2. Implement UTAM porting API.

    3. OPTIONAL. Adjust build system to a new compiler.

    4. Build time options

    Shared options

    Option name

    Description

    Affected API

    Requirements

    IOTCS_MESSAGING_THREAD_SAFETY

    Enable tread safety for Messaging API. It means that library's public API in iotcs_device.h and advanced/iotcs_messaging.h becomes thread-safe, of course except iotcs_init/iotcs_finalize calls. Also it enables diagnostic capability deviceModels/urn:oracle:iot:dcd:capability:diagnostics/testConnectivity

    iotcs_device.h:

    • iotcs_get_endpoint_id(void)

    • iotcs_activate(…)

    • iotcs_is_activated(void)

    • iotcs_register_device(…)


    advanced/iotcs_messaging.h:

    • iotcs_send(…)

    • iotcs_receive(…)

    • iotcs_request_message_free()

    • iotcs_request_dispatcher_init()

    • iotcs_request_dispatcher_finalize()

    • iotcs_request_dispatcher_dispatch()

    • iotcs_register_request_handler()

    • iotcs_unregister_request_handler_all()

    • iotcs_unregister_request_handler()

    • iotcs_get_request_handler()




    IOTCS_MESSAGE_DISPATCHER

    Enable async message dispatcher. Message dispatcher creates two threads: send and receive. Send thread transmit enqueued messages. Receive thread receive and process incoming requests in its context. Presence of Message Dispatcher enables iotcs_message_dispatcher_XXX calls from advanced/iotcs_messaging.h including iotcs_message_dispatcher_queue that send message asynchronously. iotcs_send, iotcs_receive and iotcs_request_message_free in advanced/iotcs_messaging.h will be unavailable. Also it enables message dispatcher capabilities deviceModels/urn:oracle:iot:dcd:capability:message_dispatcher/XXX.

    advanced/iotcs_messaging.h:

    • iotcs_message_dispatcher_set_delivery_callback()

    • iotcs_message_dispatcher_set_error_callback()

    • iotcs_message_dispatcher_queue()

    iotcs_device.h:

    • iotcs_init()

    • iotcs_finalize()

    IOTCS_MESSAGING_THREAD_SAFETY enabled

    IOTCS_MESSAGE_PERSISTENCE

    Enable option for async message dispatcher to save messages. Message dispatcher will save messages with IOTCS_MESSAGE_RELIABILITY_GUARANTED_DELIVERY to SQLite3 database before send. If send was successfull message dispatcher will remove it from SQLite. IOTCS_MESSAGE_PERSISTENCE is switched off for MBED platform.

    advanced/iotcs_messaging.h::

    • iotcs_message_dispatcher_queue()

    IOTCS_MESSAGE_DISPATCHER enabled

    IOTCS_ VIRTUALIZATION_SUPPORT

    Enable support of Virtualization API. It means that public API in iotcs_virtual_device.h will be available.

    iotcs_virtual_device.h:

    • iotcs_get_device_model_handle()

    • iotcs_free_device_model_handle()

    • iotcs_get_virtual_device_handle()

    • iotcs_free_virtual_device_handle()

    • iotcs_virtual_device_set_XXX()

    • iotcs_virtual_device_get_XXX()

    • iotcs_virtual_device_set_on_change()

    • iotcs_virtual_device_attribute_set_on_change()

    • iotcs_virtual_device_set_on_error()

    • iotcs_virtual_device_attribute_set_on_error()

    • iotcs_virtual_device_set_callback()

    • iotcs_virtual_device_start_update()

    • iotcs_virtual_device_finish_update()

    • iotcs_virtual_device_get_XXX_handle()

    • iotcs_virtual_device_free_XXX_handle()

    • iotcs_alert_raise()

    • iotcs_alert_set_XXX()

    • iotcs_data_submit()

    • iotcs_data_set_XXX()

    IOTCS_MESSAGING_THREAD_SAFETY enabled

    IOTCS_MESSAGE_DISPATCHER enabled

    IOTCS_ GATEWAY

    Enable support of register and activate an indirectly-connected device.

    iotcs_device.h:

    • iotcs_register_device()


    IOTCS_USE_STATIC_HEAP

    Turn on memory allocation from static memory pool instead on libc's malloc/free. Static memory pool will be IOTCS_STATIC_HEAP_SIZE bytes in size.

    util_memory.h


    IOTCS_STATIC_HEAP_SIZE

    Override static memory pool size in bytes.

    Default value: 8192

    util_memory.h


    IOTCS_USE_DRAFT_DEVICE_MODELS

    Enable creation of draft Device Models. When a registered device has device models that aren’t defined yet in Oracle IoT Cloud Service, the device can declare these models during activation. These draft device models aren’t usable until they are saved and become normal device models. When draft models are saved, the device that declared them during activation is also updated.

    iotcs_device.h:

    • iotcs_activate()


    IOTCS_USE_SERVER_TIME

    Enable static usage time from server. In that case library will take time from the server response on /iot/api/v2/oauth2/token request instead of system time.



    IOTCS_DEVICE_MODEL_DIR

    Specify to enable device model caching on local file system. Library will take device models from local IOTCS_DEVICE_MODEL_DIR file instead of server.



    IOTCS_DEBUG

    Enable debug mode

    log.h


    IOTCS_DISABLE_MQTT

    Disable mqtt support. iotcs_init check server scheme that obtained from TAM. iotcs_init will return false (unsupported server scheme warning) for mqtt server scheme if IOTCS_DISABLE_MQTT enabled.

    iotcs_device.h:

    • iotcs_init ()

    util_buffer.h


    IOTCS_DISABLE_MQTT_ACCEPTED_BYTES

    Disable mqtt accepted bytes support.

    mqtt_wrapper.h

    IOTCS_DISABLE_MQTT disabled

    IOTCS_DISABLE_HTTP

    Disable http support. iotcs_init check server scheme that obtained from TAM. iotcs_init will return false (unsupported server scheme warning) for http server scheme if IOTCS_DISABLE_HTTP enabled.

    iotcs_device.h:

    • iotcs_init ()

    util_buffer.h


    IOTCS_LONG_POLLING

    Enable the long polling implementation. The client polls the server requesting new information.  The server holds the request open until new data is available. Once available, the server responds and sends the new information. When the client receives the new information, it immediately sends another request, and the operation is repeated. This effectively emulates a server push feature.

    iotcs_device.h:

    • iotcs_init ()

    • iotcs_finalize()

    • iotcs_activate()

    • iotcs_send()

    • iotcs_receive()

    iotcs_port_ssl.h:

    • iotcs_port_ssl_connect_lp()

    • iotcs_port_ssl_disconnect_lp()

    • iotcs_port_ssl_write_lp()

    • iotcs_port_ssl_read_lp

    messaging_send_thread and messaging_recv_thread in MD

    util_buffer.h

    IOTCS_DISABLE_HTTP disabled

    (IOT-29654)

    IOTCS_DEVICE_MODEL_FILENAME_BUFFER_SIZE

    The size of buffer for device model filename in bytes

    Default value: 128



    IOTCS_CAPABILITY_RESPONSE_BUFFER_SIZE

    The size of buffer for device model capability server response in bytes

    Default value: 1024



    IOTCS_POLLING_TIMEOUT_MS

    The polling timeout in milliseconds for internal async message dispatcher

    Default value: 5000



    IOTCS_MAX_MESSAGES_FOR_SEND

    The maximum number messages to send in one request

    Default value: 5



    IOTCS_ASYNC_FAIL_REASON_BUFFER_SIZE

    The size of buffer for reason of async message send failure in bytes

    Default value: 256



    IOTCS_REQUEST_MESSAGE_BUFFER_SIZE

    The size of buffer for request message in bytes

    Default value: 512



    IOTCS_REQUEST_MESSAGE_NUMBER

    The number of request messages

    Default value: 4



    IOTCS_MAX_RESOURCE_NUMBER

    The maximum number of resource for registration

    Default value: 6



    IOTCS_SERVER_HOST_BUFFER_LENGTH

    The server host buffer length in bytes. The default value is 128 + 14, where 128 is got from TAM store and 14 is the length of string 'server.host=' + '/n' + '/0'

    Default value: 142



    IOTCS_CLIENT_ID_BUFFER_LENGTH

    The client ID buffer length in bytes. The default value is 20 + 12, where 12 is the length of string 'client.id'= + '/n' + '/0'

    Default value: 32



    IOTCS_SHARED_SECRET_BUFFER_LENGTH

    The shared secret buffer length in bytes. The default value is 50 + 15, where 15 is the length string 'client.secret' + '/n' + '/0'

    Default value: 65



    IOTCS_SIGNATURE_BUFFER_LENGTH

    The signature buffer length in bytes

    Default value: 256



    IOTCS_ASSETS_SIGNATURE_BUFFER_LENGTH

    The assets signature buffer length in bytes

    Default value: 256



    IOTCS_PUBLIC_KEY_BUFFER_LENGTH

    The public key buffer length in bytes. The default value is 350. It's enough for public key with length 2048 bits

    Default value: 350



    IOTCS_SERVER_PORT_LENGTH

    The server port length in bytes. The default value is 19, where 12 is the length of 'server.port=' + 5 is max port value '65535' + 1('/n') + 1('/0') = 19

    Default value: 19



    IOTCS_PAYLOAD_BUFFER_SIZE

    The payload buffer size in bytes. It is used for storing messages payload. The less then IOTCS_MIN_PAYLOAD_BUFFER_SIZE value is not acceptable for IOTCS_PAYLOAD_BUFFER_SIZE and set forced to IOTCS_MIN_PAYLOAD_BUFFER_SIZE in this case

    Default value: 4096



    IOTCS_MESSAGE_BUFFER_SIZE

    The message buffer size in bytes. It is used for storing of the serialised messages and server responses. The less then IOTCS_DEVICE_MODEL_URL_SIZE value is not acceptable on compile time and set forced to IOTCS_DEVICE_MODEL_URL_SIZE in this case

    Default value: 4096



    IOTCS_MQTT_SUB_TOPIC_BUFFER_SIZE

    The mqtt topic name buffer size

    Default value: 64



    IOTCS_LOG_LEVEL

    The global library log level

    Supported log levels:

    * 0 = none|no - No log

    * 1 = critical|crit - Critical log entry: unrecoverable internal error - library can't work properly anymore

    * 2 = error|err - Error log entry: recoverable error, user action is required

    * 3 = warning|warn - Warning log entry: recoverable error, no user action is required

    * 4 = info - Information log entry: not an error

    * 5 = debug|dbg - Debug log entry

    * 5 = all

    Default value: 4 (info)



    IOTCS_LOG_USE_FILE_LINE

    Enable include base file name and line number in the log entry. It is implicitly turned on by default for debug mode



    IOTCS_LOG_LEVEL_CHANNEL_CORE

    The iotcs module channel log level

    Default value: IOTCS_LOG_LEVEL



    IOTCS_LOG_LEVEL_CHANNEL_DM

    The device model channel log level

    Default value: IOTCS_LOG_LEVEL



    IOTCS_LOG_LEVEL_CHANNEL_EXT

    The external module channel log level

    Default value: IOTCS_LOG_LEVEL



    IOTCS_LOG_LEVEL_CHANNEL_JSON

    The json module channel log level

    Default value: IOTCS_LOG_LEVEL



    IOTCS_LOG_LEVEL_CHANNEL_MSG

    The messaging module channel log level

    Default value: IOTCS_LOG_LEVEL



    IOTCS_LOG_LEVEL_CHANNEL_PROTO

    The protocol module channel log level

    Default value: IOTCS_LOG_LEVEL



    IOTCS_LOG_LEVEL_CHANNEL_TAM

    The TAM module channel log level

    Default value: IOTCS_LOG_LEVEL



    IOTCS_LOG_LEVEL_CHANNEL_UTIL

    The util module channel log level

    Default value: IOTCS_LOG_LEVEL



    IOTCS_LOG_LEVEL_CHANNEL_LOG

    The logging module channel log level

    Default value: IOTCS_LOG_LEVEL



    IOTCS_LOG_LEVEL_CHANNEL_PORT_QUEUE

    The port blocking queue module channel log level

    Default value: IOTCS_LOG_LEVEL



    IOTCS_LOG_LEVEL_CHANNEL_PORT_CRYPTO

    The port crypto module channel log level

    Default value: IOTCS_LOG_LEVEL



    IOTCS_LOG_LEVEL_CHANNEL_PORT_TAM

    The port TAM module channel log level

    Default value: IOTCS_LOG_LEVEL



    IOTCS_LOG_LEVEL_CHANNEL_PORT_DIAG

    The port diagnostic module channel log level

    Default value: IOTCS_LOG_LEVEL



    IOTCS_LOG_LEVEL_CHANNEL_PORT_MEM

    The port memory module channel log level

    Default value: IOTCS_LOG_LEVEL



    IOTCS_LOG_LEVEL_CHANNEL_PORT_MUTEX

    The port mutex module channel log level

    Default value: IOTCS_LOG_LEVEL



    IOTCS_LOG_LEVEL_CHANNEL_PORT_SSL

    The port SSL module channel log level

    Default value: IOTCS_LOG_LEVEL



    IOTCS_LOG_LEVEL_CHANNEL_PORT_SYSTEM

    The port system module channel log level

    Default value: IOTCS_LOG_LEVEL



    IOTCS_LOG_LEVEL_CHANNEL_PORT_THREAD

    The port thread module channel log level

    Default value: IOTCS_LOG_LEVEL



    IOTCS_LOG_LEVEL_CHANNEL_PORT_MQTT

    The port MQTT module channel log level

    Default value: IOTCS_LOG_LEVEL



    IOTCS_REQUEST_HANDLER_THREAD_POOL_SIZE

    Number of threads created for dispatching request messages in parallel so that request handlers can be executed in parallel.

    Default value: 1



    5. Mandatory Porting API

    iotcs_port_system.h:

    void iotcs_port_platform_init()
    void iotcs_port_platform_finilize()
    Library calls iotcs_port_platform_init during library's initialization and before start using iotcs_port_system.h API and iotcs_port_platform_finilize when stopped using iotcs_port_system.h API.

    int64_t iotcs_port_get_current_time_millis(void)
    Time since epoch in milliseconds.

    const char *iotcs_port_get_os_name(void)
    OS name. E.g. "Ubuntu". Basically, any non-empty string will work.

    const char *iotcs_port_get_os_version(void)
    OS version. E.g. "14.4". Basically, any non-empty string will work.

    Used in /iot/api/v2/activation/policy?OSName=XXX&OSVersion=YYY REST API call. For more info please read REST API documentation .

    iotcs_port_queue.h:

    iotcs_port_request_queue iotcs_port_request_queue_create();
    void iotcs_port_request_queue_destroy(iotcs_port_request_queue queue);
    iotcs_result iotcs_port_request_queue_put(iotcs_port_request_queue queue, iotcs_request_message *request, int32_t timeout_ms);
    iotcs_result iotcs_port_request_queue_get(iotcs_port_request_queue queue, iotcs_request_message **request /* OUT */, int32_t timeout_ms);
    Same as message queue (see Optional for Message Dispatcher section of Optional Porting API) but stores just pointer to iotcs_request_message, not the whole structure. Queue size must be IOTCS_PORT_REQUEST_QUEUE_SIZE.

    iotcs_port_crypto.h:

    iotcs_result iotcs_port_crypto_init(void)
    void iotcs_port_crypto_finalize(void)
    Initialize/Finalize crypto methods. Library calls iotcs_port_crypto_init before start using iotcs_port_crypto.h API and iotcs_port_crypto_finalize when stopped using iotcs_port_crypto.h API.

    iotcs_result iotcs_port_crypto_encode_base64(char* output, size_t* output_length, const char* input, size_t input_length);
    iotcs_result iotcs_port_crypto_decode_base64(char* output, size_t* output_length, const char* input, size_t input_length);
    Base64 encoding/decoding.

    iotcs_result iotcs_port_md5(unsigned char *output_buffer, iotcs_port_get_next_string_func func, void *data);
    Calculate MD5 digest for strings returned by func(data). Stop when func(data) returns NULL.

    iotcs_port_ssl.h:

    iotcs_result iotcs_port_ssl_init(const char* addr, unsigned short port);
    void iotcs_port_ssl_finalize(void);

    Initialize/Finalize ssl methods. Library calls iotcs_port_ssl_init before start using
    iotcs_port_ssl.h API and iotcs_port_ssl_finalize when stopped using using iotcs_port_ssl.h API.

    iotcs_result iotcs_port_ssl_connect(void);
    iotcs_result iotcs_port_ssl_write(char* request, size_t length);
    iotcs_result iotcs_port_ssl_read(char* buffer, int len, int *bytes_read);
    iotcs_result iotcs_port_ssl_disconnect(void);

    Blocking methods for connect/write/read/disconnect. This is the order in which library calls those methods. In multithreaded configuration (if
    IOTCS_MESSAGING_THREAD_SAFETY is enabled) the whole connect/write/read/disconnect sequence is invoked in one context exclusively. This means that non thread safe implementation is acceptable.

    6. Optional Porting API

    Dynamic Memory Allocation API

    This API is used only if IOTCS_USE_STATIC_HEAP is NOT enabled. Otherwise library allocates memory from static memory pool which is IOTCS_STATIC_HEAP_SIZE bytes in size.

    iotcs_port_memory.h:

    void* iotcs_port_malloc(size_t size);
    void iotcs_port_free(void* ptr);

    Similar to libc’s malloc/free calls.

    Optional thread related API

    This API is used only if IOTCS_MESSAGING_THREAD_SAFETY build option is enabled. This option means that library's public API in iotcs_device.h and advanced/iotcs_messaging.h becomes thread-safe, of course except iotcs_init/iotcs_finalize calls.

    iotcs_port_thread.h:

    iotcs_port_thread iotcs_port_thread_create(iotcs_port_thread_func func_ptr);
    Create new thread and start it.

    void iotcs_port_thread_join(iotcs_port_thread thread);
    Gracefully join given thread - no forced termination.

    void iotcs_port_thread_cleanup(void);
    Called in thread context before thread finish execution. This could be used to release automatically allocated thread local resources, like openSSL does. In most cases this method could be NO-OP.

    int iotcs_port_sleep_millis(int32_t timeout_ms);
    Block current thread for timeout_ms milliseconds. Used to support
    testConnectivity of urn:oracle:iot:dcd:capability:diagnostics device model. For more info please read Device Model documentation and Capability documentation .

    iotcs_port_mutex.h:

    iotcs_port_mutex iotcs_port_mutex_create(void);
    void iotcs_port_mutex_destroy(iotcs_port_mutex mutex);
    void iotcs_port_mutex_lock(iotcs_port_mutex mutex);
    void iotcs_port_mutex_unlock(iotcs_port_mutex mutex);

    Mutex API.

    Optional diagnostic capability porting API

    This API is used to implement info of urn:oracle:iot:dcd:capability:diagnostics device model. For more info please read Device Model documentation and Capability documentation . This API is used only if IOTCS_USE_DIAGNOSTIC_CAPABILITY build option is enabled.

    Library doesn't interpret anyhow returned values, it just pass the values "as is" to IoT server on diagnostic info request.

    iotcs_port_diagnostics.h:

    int iotcs_port_get_ip_address(char* buffer, int buf_len);
    int iotcs_port_get_mac_address(char* buffer, int buf_len);

    Write IP/MAC address ('\0' terminated string) to a given buffer. Return number of written bytes excluding '\0' character or negative number in case of error or buffer overflow.

    int64_t iotcs_port_get_start_time(void);
    Return device start time in milliseconds since epoch or negative number in case of error.

    const char* iotcs_port_get_version(void);
    Address of static buffer with '\0' terminated device version string.

    uint64_t iotcs_port_get_total_disk_space(void);
    Return total disk space in bytes.

    uint64_t iotcs_port_get_free_disk_space(void);
    Return free disk space in bytes

    Optional for Message Dispatcher

    Only if IOTCS_MESSAGE_DISPATCHER build option is enabled.

    iotcs_port_queue.h:

    iotcs_port_message_queue iotcs_port_message_queue_create();
    void iotcs_port_message_queue_destroy(iotcs_port_message_queue queue);
    create/destroy message queue. Queue size must be IOTCS_PORT_MESSAGE_QUEUE_SIZE. Element size is sizeof(iotcs_message). If IOTCS_MESSAGING_THREAD_SAFETY is defined then queue must be blocking.

    iotcs_result iotcs_port_message_queue_put(iotcs_port_message_queue queue, iotcs_message *message, int32_t timeout_ms);
    iotcs_result iotcs_port_message_queue_get(iotcs_port_message_queue queue, iotcs_message *message /* OUT */, int32_t timeout_ms);
    Put/Get message to/from queue coping sizeof(iotcs_message) bytes. If IOTCS_MESSAGING_THREAD_SAFETY is defined and queue is full/empty then these calls must use timeout_ms milliseconds timeout waiting for free space/message. If timeout is expired then error must be returned. If IOTCS_MESSAGING_THREAD_SAFETY is NOT defined then timeout_ms could be ignored treating it like zero timeout.

    int iotcs_port_message_queue_count(iotcs_port_message_queue queue);
    Returns number of queued messages in
    queue. Library doesn't interpret anyhow returned values, it just pass the values "as is" to IoT server on counters request of urn:oracle:iot:dcd:capability:message_dispatcher device model. For more info please read Device Model documentation and Capability documentation .

    Returned value is used to calculate message dispatcher load factor as follow: count/size. Load factor is returned as a load field value of counters request of urn:oracle:iot:dcd:capability:message_dispatcher device model.

    Optional for Long Polling support

    Only if IOTCS_LONG_POLLING build option is enabled.

    iotcs_port_ssl.h:

    Without Long polling support library use just one network socket at a time. Long polling use one additional lp (long-polling) socket.

    iotcs_result iotcs_port_ssl_connect_lp(int32_t timeout_ms);
    iotcs_result iotcs_port_ssl_disconnect_lp(void);
    iotcs_result iotcs_port_ssl_write_lp(char* request, size_t length);
    iotcs_result iotcs_port_ssl_read_lp(char* buffer, int len, int *bytes_read);

    Same as corresponding methods without _lp suffix. Blocking methods for connect/write/read/disconnect. This is the order in which library calls those methods. In multithreaded configuration if
    IOTCS_MESSAGING_THREAD_SAFETY is enabled) the whole connect/write/read/disconnect sequence is invoked in one context exclusively. It is important to note that e.g. iotcs_port_ssl_connect_lp and iotcs_port_ssl_connect could happen concurrently. Implementation must be robust to this.

    Optional for automatic device time adjustment

    Only if IOTCS_USE_SERVER_TIME build option is enabled.

    iotcs_port_system.h:

    Build time option IOTCS_USE_SERVER_TIME allows to fetch time from server side (works only for HTTPS protocol).

    OAuth2 protocol requires time on server and client to be in sync. Time difference could lead to inability to get access token which is required for every HTTPS request.

    When IOTCS_USE_SERVER_TIME is enabled and request to get access token failed, library parses server response, extracts server time from it and passes result to iotcs_port_set_current_time call.

    Implementation doesn't have to adjust hardware RTC, it could just calculate a time difference and adjust return value of iotcs_port_get_current_time_millis call by the difference.

    void iotcs_port_set_current_time(int64_t seconds);
    Here seconds is a time in seconds since epoch.

    7. Porting MQTT support

    To support MQTT protocol IoT C client library uses open-source MQTT client “Paho embedded”. This client has its own porting layer so in order to support MQTT one shall implement porting layer of “Paho embedded” together with IoT C CL bindings. Let’s consider both APIs.

    Paho Embedded porting layer

    The implementation shall be in file iotcs_port_mqtt.h. The file is included in sources using this particular name so the exact name is required. This file shall define two new types Timer and Network. E.g.:

    typedef struct my_timer Timer;

    typedef struct my_network Network;

    Generally Timer is platform timer handle and iotcs_port_mqtt.h file shall define following API for Timer type:

    void InitTimer(Timer*t)

    Reset and clears given timer t.

    void countdown_ms(Timer*t, unsigned int timeout_ms);

    Starts timer t that shell expire after timeout_ms milliseconds.

    void countdown(Timer*t, unsigned int timeout_sec);

    Starts timer t that shell expire after timeout_sec seconds.

    char expired(Timer*t);

    Returns non-zero value is given t has already expired.

    int left_ms(Timer*t);

    Returns remain milliseconds until timer t expiration or zero is timer has already expired.

    Implementation of these methods must be available for linker during linkage of course.

    E.g. If platform has just one method to get value of monotonic microseconds ticker (us_ticker()) then implementation could be as follow:

    typedef uint32_t Timer;

    void InitTimer(Timer* timer) { (void)timer; /* no action required */ }

    void countdown_ms(Timer* timer, unsigned int timeout) {

    *((uint32_t*)timer) = us_ticker() + (uint32_t)(timeout * 1000); /* to us */

    }

    void countdown(Timer* timer, unsigned int timeout) {

    *((uint32_t*)timer) = us_ticker() + (uint32_t)(timeout * 1000000); /* to us */

    }


    char expired(Timer* timer) {

    return (char) ( (int32_t)(*((uint32_t*)timer) - us_ticker()) < 0);

    }

    int left_ms(Timer* timer) {

    int32_t remain = (*((uint32_t*)timer) - us_ticker()) / 1000;

    return remain > 0 ? remain : 0;

    }

    The Network type shell be a struct with at least two function pointers:

    typedef struct my_network Network;

    struct my_network

    {

    int (*mqttread) (Network*, unsigned char*, int, int);

    int (*mqttwrite) (Network*, unsigned char*, int, int);

    };

    Network type is a handle for establishing/closing network connection and reading/writing data to it. Paho library uses Network instance (e.g. n) as follow:

    Network *n;

    n->mqttread(n, buf, buf_len, timeout_ms);

    n->mqttwrite(n, buf, buf_len, timeout_ms);

    Here,

    int (*mqttread) (Network*network, unsigned char*buffer, int buffer_len, int timeout_ms);

    Method shell read as much as possible bytes from network to a buffer buffer until buffer_len bytes is read or timeout_ms timeout has expired. Method returns number of read bytes or negative number in case of error.

    int (*mqttwrite) (Network* network, unsigned char* buffer, int buffer_len, int timeout_ms);

    Method shell write data from a buffer buffer to network to until buffer_len bytes is written or timeout_ms timeout has expired. Method returns number of written bytes or negative number in case of error.

    IoT Library Paho bindings

    The Library needs just two abstractions for connecting and disconnecting network socket. Following methods are called by the Library in order to do so:

    Network* iotcs_port_mqtt_network_connect(char*url, int port);

    Establishes SSL connection with given url and port. Returns non-NULL pointer with connected network.

    void iotcs_port_mqtt_network_disconnect();

    Closes SSL connection previously established with iotcs_port_mqtt_network_connect call.

    These methods are called from one thread only and it is guaranteed that only one connection is in use at a time. Thus iotcs_port_mqtt_network_disconnect doesn’t need to get particular Network address.

    Paho Embedded port example

    There are existing ports of paho embedded to posix and mbed platform that could be used as an example.

    Posix port could be found in source bundle from Posix CL release in directory ./iotcs/csl/posix/src/posix/ , files: iotcs_port_mqtt.c, iotcs_port_mqtt.h, iotcs_port_ssl.c.

    Mbed port could be found in source bundle from Mbed CL release in directory ./iotcs/csl/mbed/src/mbed/ , files: iotcs_port_mqtt.c, iotcs_port_mqtt.h, iotcs_port_ssl.cpp.

    8. UTAM porting API

    Library needs a TAM module that provides trusted_assets_manager.h API. This API gives library access to IoT server URL, endpoint id, endpoint public key and encryption using shared secret and private key.

    There is an internal TAM implementation that supports two modes: Default TAM (obsolete and deprecated - kept for compatibility) and Unified TAM (unified format for all client libraries).

    iotcs_port_tam.h:

    void iotcs_port_tam_init (void);
    Initializes porting layer for default TAM.

    void iotcs_port_tam_finalize(void);
    Clears all the resources allocated so far.

    iotcs_result iotcs_port_tam_set_ts_password(const char* trust_store_password);
    Save given password for later use.

    size_t iotcs_port_tam_secret_hash(IOTCS_TYPE_SHA type, const unsigned char* key, size_t key_length, const unsigned char* content, size_t content_length, unsigned char *hash, size_t hash_maxsize);
    Generates secret hash used for authentication and to get access token.

    iotcs_result iotcs_port_tam_unified_credentials_decrypt(unsigned char *out, size_t* out_len, const unsigned char* in, size_t in_len, const unsigned char* iv);
    Decrypts the data from unified trusted assets store. Function uses password given with iotcs_port_tam_set_ts_password call. Expands the password into 16 bytes using PKCS#5 PBKDF2 SHA1 HMAC with 10000 iterations. And finally decrypt the data with 128 bit AES Cipher Block Chaining (CBC) and PKCS#5 padding. iv argument is used as initialization vector in decryption process.

    iotcs_result iotcs_port_tam_unified_credentials_encrypt(unsigned char *out, size_t* out_len, const unsigned char* in, size_t in_len, const unsigned char* iv);
    Similar to iotcs_port_tam_unified_credentials_decrypt but does inverse transformation.

    size_t iotcs_port_tam_get_trust_anchors_count(void);
    Get number of trusted certificates

    const char* iotcs_port_crypto_get_trust_anchor_certificate(size_t index, size_t *len);
    Get certificate in index position in DER format.

    iotcs_result iotcs_port_crypto_serialize_private_key(unsigned char *buf, size_t buf_length, size_t *written);
    Serialize private key in PKCS#8 format into given buffer.

    iotcs_result iotcs_port_crypto_deserialize_private_key(unsigned char *buf, size_t buf_length);
    Load private key (and public) in PKCS#8 format from given buffer.

    iotcs_result iotcs_port_ tam_generate_keypair(size_t key_size, unsigned char* public_out, size_t* public_length);
    Generates RSA key pair and return public key in DER format.

    size_t iotcs_port_tam_sign_with_private_key(IOTCS_TYPE_SHA type, const unsigned char* input, size_t input_length, char* signature, size_t signature_maxsize);
    Sign given input with generated private key (in call
    iotcs_port_tam_generate_keypair) or loaded private key (in call iotcs_port_crypto_deserialize_private_key).

    Default TAM support methods (Obsolete and Deprecated)

    There is no need to implement these methods as long as no Default TAM support is required.

    iotcs_port_tam.h:

    iotcs_result iotcs_port_tam_decode_shared_secret(char* encrypted_shared_secret, char *shared_secret, int shared_secret_length);
    iotcs_result iotcs_port_tam_check_file_signature(const char* file, size_t signed_data_length, const char* signature);
    iotcs_result iotcs_port_tam_load_trust_store(const char* trust_store_path);
    iotcs_result iotcs_port_tam_load_trust_store_achors(const unsigned char* trust_store_anchors, int length);

    iotcs_result iotcs_port_tam_credentials_load(const char *client_id, char *endpoint_id, size_t *endpoint_length, unsigned char *public_key, size_t *public_key_len);
    iotcs_result iotcs_port_tam_credentials_store(void);

    9. Custom TAM

    If Unified TAM doesn’t meet security requirements or target device doesn’t have local file system or for some other reason user could use his own Custom TAM implementation instead of UTAM. Custom TAM must provide TAM API to C CL library. TAM API that could be found in source bundle in file ./iotcs/csl/posix/src/shared/trusted_assets_manager/iotcs_tam.h (path for posix release). UTAM implementation could be used as an example (could be found in source bundle in file ./iotcs/csl/posix/src/shared/trusted_assets_manager/iotcs_tam.c - path for posix release).

    10. Examples of porting

    There are existing ports that could serve as a good example. Probably the most demonstrative port example is the Posix port. This port uses:

    Another good example of porting, but this time to embedded system, is port to frdm-k64f board which is called “Mbed” port. Embedded systems tends to be a resource constrained devices with limited amount of memory and CPU power and this fact has an impact on implementation. This port uses:

    11. Build system adjustment

    For building the library on a new platform probably it is worth to reuse existing build system. This section will describe required steps to do so.

    Please note that all the instructions is this guide are for posix release, for other releases please substitute “posix” directory with the release platform name. Also this guide might not include all needed steps.

    How to add custom porting layer

    This is the instruction how to build the library with an implementation for a new porting layer.

    1. Open iotcs/csl/posix/make/config/paths.cfg

    2. Set the path to directory with port sources (<custom_port_src>) as a PORT_SRC_DIR variable value

    3. Create your implementation for iotcs/csl/posix/src/include/port/iotcs_port_XXX.h API in < custom_port_src >/<custom_port_XXX.c> files. iotcs_port_tam.h could be not implemented.

    4. Copy iotcs/csl/posix/src/include/port/iotcs_port_tam.h into <custom_port_src> folder. This needed to minimize changes in build system.



    Otherwise you should change /csl/posix/make/posix/impl.mk at line 13 that contained target to compile port TAM sources like:

    # Compile port TAM source files into .o files

    $(PORT_TAM_OBJ_DIR)/%.o: $(SRC_DIR)/$(PORT)/%.c

    $(A)mkdir -p $(dir $@)

    $(A)$(CC) $(CPPFLAGS) $(CPP_OPTS) $(CFLAGS) $(C_OPTS) $(PORT_FLAGS) $(PORT_OPTS) $(TAM_FLAGS) $(TAM_OPTS) $(OBJECT_FILE_NAME_OPT)$@ $<

    And add include in CPPINCLUDES variable in iotcs/csl/posix/make/common.mk at line 115:

    CPPINCLUDES=-I$(PORT_INC_DIR) -I$(INC_DIR) -I$(SHARED_DIR) -I$(OUT_LIB_PATH)

    -I$(SRC_DIR)/$(PORT)

    1. (optional) Add target to compile c++ files in file/csl/posix/make/posix/impl.mk:

      $(PORT_OBJ_DIR)/%.o: $(PORT_SRC_DIR)/%.cpp

      $(A)mkdir -p $(dir $@)

      $(A)$(CC) $(CPPFLAGS) $(CPP_OPTS) $(CXXFLAGS) $(CXX_OPTS) $(PORT_FLAGS) $(PORT_OPTS) $(OBJECT_FILE_NAME_OPT)$@ $<

    2. Build library

    After these changes you could build library as usual, but library will have custom porting layer in it. In addition, you can build library with only porting layer.

    Generate library with only porting layer:

    make .library_porting_impl

    How to add new compiler

    If target platform’s compiler is not GCC then in order to support new compiler in the build system it is required to follow these steps:

    1. Go to /iotcs/csl/posix/make/cc

    2. Copy gcc.cfg to <compiler>.cfg

    3. (optional) Set default values for custom arguments. For example,

      TOOLS_PREFIX?=/<compiler_path>/bin

      OUT_CC_SUBDIR?=<compiler>

    4. Set internal variables

      CC=$(TOOLS_PREFIX)<compiler>

      CXX=$(TOOLS_PREFIX)<compiler for c++>

      AS=$(TOOLS_PREFIX) <compiler for assembler files>

      LD=$(TOOLS_PREFIX) <linker>

      AR=$(TOOLS_PREFIX)< archiver>

      OBJCPY=$(TOOLS_PREFIX) <object copy tool>

    5. Go to /iotcs/csl/posix/make and execute next command:

    make CC_CFG=<compiler>

    How to add custom TAM

    Custom TAM:

    1. Create your implementation for iotcs/csl/posix/src/shared/trusted_assets_manager/iotcs_tam.h API in file < custom_tam _path>/<custom_tam.c>

    2. Open iotcs/csl/posix/make/common.mk at line 313 that contained target to compile TAM sources

      # targets for compile TAM sources

      $(TAM_OBJ_DIR)/%.o: $(SHARED_DIR)/%.c

      $(A)mkdir -p $(dir $@)

      $(A)$(CC) $(CPPFLAGS) $(CPP_OPTS) $(CFLAGS) $(C_OPTS) $(TAM_FLAGS) $(TAM_OPTS) $(OBJECT_FILE_NAME_OPT)$@ $<

    3. Change this target:

    # targets for compile TAM sources

    $(TAM_OBJ_DIR)/%.o: < custom_tam _path>/<custom_tam.c>

    $(A)mkdir -p $(dir $@)

    $(A)$(CC) $(CPPFLAGS) $(CPP_OPTS) $(CFLAGS) $(C_OPTS) $(TAM_FLAGS) $(TAM_OPTS) $(OBJECT_FILE_NAME_OPT)$@ $<

    < custom_tam _path> must be either an absolute path or a path relative to the iotcs/csl/posix folder ($(PROJ_DIR) variable)

    Custom port TAM:

    If required it is possible to support porting layer for a custom TAM.

    1. Create your implementation for iotcs/csl/posix/src/include/port/iotcs_port_tam.h API in file < custom_port_tam _path >/<custom_port_tam.c>

    2. Open iotcs/csl/posix/make/posix/impl.mk at line 13 that contained target to compile port TAM sources

      # Compile port TAM source files into .o files

      $(PORT_TAM_OBJ_DIR)/%.o: $(PORT_SRC_DIR)/%.c

      $(A)mkdir -p $(dir $@)

      $(A)$(CC) $(CPPFLAGS) $(CPP_OPTS) $(CFLAGS) $(C_OPTS) $(PORT_FLAGS) $(PORT_OPTS) $(TAM_FLAGS) $(TAM_OPTS) $(OBJECT_FILE_NAME_OPT)$@ $<

    3. Change this target:

    # Compile port TAM source files into .o files

    $(PORT_TAM_OBJ_DIR)/%.o: < custom_port_tam _path >/<custom_port_tam.c>

    $(A)mkdir -p $(dir $@)

    $(A)$(CC) $(CPPFLAGS) $(CPP_OPTS) $(CFLAGS) $(C_OPTS) $(PORT_FLAGS) $(PORT_OPTS) $(TAM_FLAGS) $(TAM_OPTS) $(OBJECT_FILE_NAME_OPT)$@ $<

    < custom_port_tam _path > must be either an absolute path or a path relative to the iotcs/csl/posix folder ($(PROJ_DIR) variable)

    After these changes you could build library as usual, but library will have custom TAM in it. In addition, you can build only TAM library.

    Generate only TAM library:

    make .tam_library_impl