Yuma Developer Manual

YANG-Based Unified Modular Automation Tools

Server Instrumentation Library Development

Version 1.15

Last Updated: July 20, 2011

Table Of Contents

1 Preface

1.1 Legal Statements

Copyright 2009 - 2011,  Andy Bierman,  All Rights Reserved.

1.2 Additional Resources

This document assumes you have successfully set up the software as described in the printed document:

Yuma  Installation Guide

Yuma Quickstart Guide

Other documentation includes:

Yuma User Manual

Yuma  netconfd  Manual

Yuma  yangcli Manual

Yuma  yangdiff Manual

To obtain additional support you may join the yuma-users group on sourceforge.net and send email to this e-mail address:

yuma-users@lists.sourceforge.net

The SourceForge.net Support Page for Yuma can be found at this WEB page:

http://sourceforge.net/projects/yuma/support

There are several sources of free information and tools for use with YANG and/or NETCONF.

The following section lists the resources available at this time.

1.2.1 WEB Sites

• Netconf Central

  • http://www.netconfcentral.org/

  • Yuma Home Page

    • Free information on NETCONF and YANG, tutorials, on-line YANG module validation and documentation database

• Yuma SourceFource OpenSource Project

  • http://sourceforge.net/projects/yuma/

    • Download Yuma source and binaries; project forums and help

• Yang Central

  • http://www.yang-central.org

  • Free information and tutorials on YANG, free YANG tools for download

• NETCONF Working Group Wiki Page

  • http://trac.tools.ietf.org/wg/netconf/trac/wiki

  • Free information on NETCONF standardization activities and NETCONF implementations

• NETCONF WG Status Page

  • http://tools.ietf.org/wg/netconf/

  • IETF Internet draft status for NETCONF documents

• libsmi Home Page

  • http://www.ibr.cs.tu-bs.de/projects/libsmi/

  • Free tools such as smidump, to convert SMIv2 to YANG

1.2.2 Mailing Lists

• NETCONF Working Group

  • http://www.ietf.org/html.charters/netconf-charter.html

  • Technical issues related to the NETCONF protocol are discussed on the NETCONF WG mailing list.  Refer to the instructions on the WEB page for joining the mailing list.

• NETMOD Working Group

  • http://www.ietf.org/html.charters/netmod-charter.html

  • Technical issues related to the YANG language and YANG data types are discussed on the NETMOD WG mailing list.  Refer to the instructions on the WEB page for joining the mailing list.

1.3  Conventions Used in this Document

The following formatting conventions are used throughout this document:

Documentation Conventions

2 Software Overview

2.1 Introduction

Refer to section 3 of the Yuma User Manual for a complete introduction to Yuma Tools.

This section focuses on the software development aspects of NETCONF, YANG, and the netconfd server.

2.1.1 Intended Audience

This document is intended for developers of server instrumentation library software, which can be used with the programs in the Yuma suite.  It covers the design and operation of the netconfd server, and the development of server instrumentation library code, intended for use with the netconfd server.

2.1.2 What does Yuma Do?

The Yuma Tools suite provides automated support for development and usage of network management information.  Refer to the Yuma User Guide for an introduction to the YANG data modeling language and the NETCONF protocol.

This section describes the Yuma development environment and the basic tasks that a software developer needs to perform, in order to integrate YANG module instrumentation into a device.

This manual contains the following information:

• Yuma Development Environment

• Yuma Runtime Environment

• Yuma Source Code Overview

• Yuma Server Instrumentation Library Development Guide

Yuma Tools programs are written in the C programming language, using the 'gnu99' C standard, and should be easily integrated into any operating system or embedded device that supports the Gnu C compiler.

2.1.3 What is a Yuma Root?

The Yuma Tools programs will search for some types of files in default locations

• YANG Modules:  The 'modules' sub-directory is used as the root of the YANG module library.

• Client Scripts: The yangcli program looks in the 'scripts' sub-directory for user scripts.

• Program Data: The yangcli and netconfd programs look for saved data structures in the 'data' sub-directory.

2.1.4 Searching Yuma Roots

1) $HOME Directory

The first Yuma root checked when searching for files is the directory identified by the $HOME environment variable.  If a '$HOME/modules', '$HOME/data'. and/or '$HOME/scripts' directory exists, then it will be checked for the specified file(s).

2) The $YUMA_HOME Directory

The second Yuma root checked when searching for files is the directory identified by the $YUMA_HOME environment variable.  This is usually set to private work directory, but a shared directory could be used as well.  If a '$YUMA_HOME/modules', '$YUMA_HOME/data'. and/or '$YUMA_HOME/scripts' directory exists, then it will be checked for the specified file(s).

3) The $YUMA_INSTALL Directory

The last Yuma root checked when searching for files is the directory identified by the $YUMA_INSTALL environment variable.  If it is not set, then the default value of '/usr/share/yuma' is used instead. This is usually set to the public directory where all users should find the default modules.  If a '$YUMA_INSTALL/modules', '$YUMA_INSTALL/data'. and/or '$YUMA_INSTALL/scripts' directory exists, then it will be checked for the specified file(s).

2.1.5 What is a SIL?

A SIL is a Server Instrumentation Library.  It contains the 'glue code' that binds YANG content (managed by the netconfd server), to your networking device, which implements the specific behavior, as defined by the YANG module statements.

The netconfd server handles all aspects of the NETCONF protocol operation, except data model semantics that are contained in description statements.  The server uses YANG files directly, loaded at boot-time or run-time, to manage all NETCONF content, operations, and notifications.

Callback functions are used to hook device and data model specific behavior to database objects and RPC operations.  The yangdump program is used to generate the initialization, cleanup, and 'empty' callback functions for a particular YANG module.  The callback functions are then completed (by you), as required by the YANG module semantics.  This code is then compiled as a shared library and made available to the netconfd server.  The 'load' command (via CLI, configuration file, protocol operation) is used (by the operator) to activate the YANG module and its SIL.

2.1.6 Basic Development Steps

The steps needed to create server instrumentation for use within Yuma are as follows:

• Create the YANG module data model definition, or use an existing YANG module.

• • Validate the YANG module with the yangdump program and make sure it does not contain any errors.  All warnings should also be examined to determine if they indicate data modeling bugs or not.

• • Example toaster.yang

   

• Make sure the $YUMA_HOME environment variable is defined, and pointing to your Yuma development tree.

• Create a SIL development subtree

• • Generate the directory structure and the Makefile with the make_sil-_ir script, installed in the /usr/bin directory.  This step will also call yangdump to generate the initial H and C files file the SIL.

     

  • Example: mydir> make_sil_dir toaster

   

• Use your text editor to fill in the device-specific instrumentation for each object, RPC method, and notification.  Almost all possible NETCONF-specific code is either handled in the central stack, or generated automatically. so this code is responsible for implementing the semantics of the YANG data model.

• Compile your code

  • Use the 'make' command in the SIL 'src' directory.  This should generate a library file in the SIL 'lib' directory.

  • Example:  mydir/toaster/src> make

   

• Install the SIL library so it is available to the netconfd server.

  • Use the 'make install' command in the SIL 'src' directory.

  • Example:  mydir/toaster/src> make install

   

• Run the netconfd server (or build it again if linking with static libraries)

• Load the new module

  • Be sure to add a 'load' command to the configuration file if the module should be loaded upon each reboot.

  • yangcli Example: load toaster

   

• The netconfd server will load the specified YANG module and the SIL and make it available to all sessions.

2.2 Yuma Source Files

This section describes the files that are contained in the yuma-source package.

The important C include files are copied into /usr/include/yuma when the yuma-dev package is installed.  The full set of installation sources is installed in /usr/share/yuma/src, if the yuma-source package is installed. Yuma tools will check the $YUMA_HOME/src sub-tree before checking this default installation location.  This allows a working copy of the Yuma sources to be used instead of the installation copy, in case it has been modified for a particular embedded system (for example).

This section lists the files that are included within the netconf/src directory.

2.2.1 src/ncx Directory

This directory contains the code that is used to build the libncx.so binary shared library that is used by all Yuma Tools programs.  It handles many of the core NETCONF/YANG data structure support, including all of the YANG/YIN, XML,  and XPath processing.  The following table describes the purpose of each file.  Refer to the actual include file (e.g., ncx.h in /usr/include/yuma) for more details on each external function in each C source module.

src/ncx C Modules

2.2.2 src/platform Directory

This directory contains platform support include files and Makefile support files.  It is used by all Yuma C modules to provide an insulating layer between Yuma programs and the hardware platform that is used.  For example the m__getMem, m__getObj, and m__freeMem macros are used instead of malloc and free functions directly.

The following table describes the files that are contained in this directory:

src/platform Files

2.2.3 src/agt Directory

This directory contains the NETCONF server implementation and built-in module SIL code.  A static library called libagt.a is built and statically linked within the netconfd program.

The following table describes the C modules contained in this directory:

src/agt C Modules

2.2.4 src/mgr Directory

This module contains the NETCONF client support code.  It handles all the basic NETCONF details so a simple internal API can be used by NETCONF applications such as yangcli.   A static library called libmgr.a is built and statically linked within the yangcli program.

The following table describes the C modules contained in this directory:

src/mgr C Modules

2.2.5 src/subsys Directory

This directory contains the netconf-subsystem program.  This is a thin-client application that just transfers input and output between the SSH server and the NETCONF server.  It contains one C source module called netconf-subsystem.  This is a stand-alone binary that is part of the yuma-server package.  It is installed in the /usr/sbin/ directory.

2.2.6 src/netconfd Directory

This directory contains the netconfd program, which implements the NETCONF server. It contains one C module called netconfd, which defines the NETCONF server 'main' function.  This is a stand-alone binary that is part of the yuma-server package.  It is installed in the /usr/sbin/ directory.

2.2.7 src/yangcli Directory

This directory contains the yangcli program, which is the Yuma NETCONF client program.  This is a stand-alone binary that is part of the yuma-client package.  It is installed in the /usr/bin/ directory.

The following table describes the C modules contained in this directory:

src/yangcli C Modules

2.2.8 src/yangdiff Directory

This directory contains the yangdiff program, which is the Yuma YANG module compare program.  This is a stand-alone binary that is part of the yuma-client package.  It is installed in the /usr/bin/ directory.

The following table describes the C modules contained in this directory:

src/yangdiff

2.2.9 src/yangdump Directory

This directory contains the yangdump program, which is the Yuma YANG compiler program.  This is a stand-alone binary that is part of the yuma-client package.  It is installed in the /usr/bin/ directory.

The following table describes the C modules contained in this directory:

src/yangdump C Modules

2.3 Server Design

This section describes the basic design used in the netconfd server.

Initialization:

The netconfd server will process the YANG modules, CLI parameters, config file parameters, and startup device NETCONF database, then wait for NETCONF sessions.

ncxserver Loop:

The SSH2 server will listen for incoming connections which request the 'netconf' subsystem.

When a new session request is received, the netconf-subsystem program is called, which opens a local connection to the netconfd server, via the ncxserver loop.  NETCONF <rpc> requests are processed by the internal NETCONF stack.  The module-specific callback functions (blue boxes) can be loaded into the system at build-time or run-time.  This is the device instrumentation code, also called a server implementation library (SIL).  For example, for libtoaster, this is the code that controls the toaster hardware.

Cleanup:

If the <shutdown> or <reboot> operations are invoked, then the server will cleanup.  For a reboot, the init cycle is started again, instead of exiting the program.

2.3.1 YANG Native Operation

Yuma uses YANG source modules directly to implement NETCONF protocol operations automatically within the server.  The same YANG parser is used by all Yuma programs.  It is located in the 'ncx' source directory (libncx.so).  There are several different parsing modes, which is set by the application.

In the 'server mode', the descriptive statements, such as 'description' and 'reference' are discarded upon input.  Only the machine-readable statements are saved.  All possible database validation, filtering, processing, initialization, NV-storage, and error processing is done, based on these machine readable statements.

For example, in order to set the platform-specific default value for some leaf, instead of hard-coded it into the server instrumentation, the default is stored in YANG data instead.  The YANG file can be altered, either directly (by editing) or indirectly (via deviation statements), and the new or altered default value specified there.

In addition, range statements, patterns, XPath expressions, and all other machine-readable statements are all processed automatically, so the YANG statements themselves are like server source code.

YANG also allows vendor and platform-specific deviations to be specified, which are like generic patches to the common YANG module for whatever purpose needed.  YANG also allows annotations to be defined and added to YANG modules, which are specified with the 'extension' statement.  Yuma uses some extensions to control some automation features, but any module can define extensions, and  module instrumentation code can access these annotation during server operation, to control device behavior.

There are CLI parameters that can be used to control parser behavior such as warning suppression, and protocol behavior related to the YANG content, such as XML order enforcement and NETCONF protocol operation support.  These parameters are stored in the server profile, which can be customized for each platform.

2.3.2 YANG Object Tree

The YANG statements found in a module are converted to internal data structures.

For NETCONF and database operations, a single tree of obj_template_t data structures is maintained by the server.  This tree represents all the NETCONF data that is supported by the server.  It does not represent any actual data structure instances.  It just defines the data instances that are allowed to exist on the server.

Raw YANG vs. Cooked YANG:

Some of the nodes in this tree represent the exact YANG statements that the data modeler has used, such as 'augment', 'refine', and 'uses', but these noeds are not used directly in the object tree.  They exist in the object tree, but they are processed to produce a final set of YANG data statements, translated into 'cooked' nodes in the object tree.  If any deviation statements are used by server implementation of a YANG data node (to change it to match the actual platform implementation of the data node), then these are also 'patched' into the cooked YANG nodes in the object tree.

2.3.3 YANG Data Tree

A YANG data tree represents the instances of 1 or more of the objects in the object tree.

Each NETCONF database is a separate data tree.  A data tree is constructed for each incoming message as well.  The server has automated functions to process the data tree, based on the desired NETCONF operation and the object tree node corresponding to each data node.

Every NETCONF node (including database nodes) are distinguished with XML Qualified Names (QName).  The YANG module namespace is used as the XML namespace, and the YANG identifier is used as the XML local name.

Each data node contains a pointer back to its object tree schema node.  The value tree is comprised of the val_value_t structure.  Only real data is actually stored in the value tree.  For example, there are no data tree nodes for choices and cases.  These are conceptual layers, not real layers, within the data tree.

The NETCONF server engine accesses individual SIL callback functions through the data tree and object tree.  Each data node contains a pointer to its corresponding object node.

Each data node may have several different callback functions stored in the object tree node. Usually, the actual configuration value is stored in the database,  However, virtual data nodes are also supported.  These are simply placeholder nodes within the data tree, and usually used for non-configuration nodes, such as counters.  Instead of using a static value stored in the data node, a callback function is used to retrieve the instrumentation value each time it is accessed.

2.3.4 Service Layering

All of the major server functions are supported by service layers in the 'agt' or 'ncx' libraries:

• Memory management: macros in platform/procdefs.h are used instead of using direct heap functions. The macros m__getMem or m__getObj are used by Yuma code to allocate memory.  Both of these functions increment a global counter called malloc_count.  The macro m__free is used to delete all malloced memory.  This macro increments a global counter called free_count.  When a Yuma program exists, it checks if malloc_count equals free_count, and if not, generates an error message.  If this occurs, the MEMTRACE=1 parameter can be added to the make command to activate 'mtrace' debugging.

• Queue management: APIs in ncx/dlq.h are used for all double-linked queue management.

• XML namespaces: XML namespaces (including YANG module namespaces) are managed with functions in ncx/xmlns.h.  An internal 'namespace ID is used internally instead of the actual URI.

• XML parsing: XML input processing is found in ncx/xml_util.h data structures and functions.

• XML message processing: XML message support is found in ncx/xml_msg.h data structures and functions.

• XML message writing with access control: XML message generation is controlled through API functions located in ncx/xml_wr.h.  High level (value tree output) and low-level (individual tag output) XML output functions are provided, which hide all namespace, indentation, and other details.  Access control is integrated into XML message output to enforce the configured data access policies uniformly for all RPC operations and notifications.  The access control model cannot be bypassed by any dynamically loaded module server instrumentation code.

• XPath Services: All NETCONF XPath filtering, and all YANG XPath-based constraint validation, is handled with common data structures and API functions.  The XPath 1.0 implementation is native to the server, and uses the object and value trees directly to generate XPath results for NETCONF and YANG purposes.  NETCONF uses XPath differently than XSLT, and libxml2 XPath processing is memory intensive.  These functions are located in ncx/xpath.h, ncx/xpath1.h, and ncx/xpath_yang.h.  XPath filtered <get> responses are generated in agt/agt_xpath.c.

• Logging service: Encapsulates server output to a log file or to the standard output, filtered by a configurable log level.  Located in ncx/log.h.  In addition, the macro SET_ERROR() in ncx/status.h is used to report programming errors to the log.

• Session management: All server activity is associated with a session.  The session control block and API functions are located in ncx/ses.h.  All input, output, access control, and protocol operation support is controlled through the session control block (ses_cb_t).

• Timer service: A periodic timer service is available to SIL modules for performing background maintenance within the main service loop.  These functions are located in agt/agt_timer.h.

• Connection management: All TCP connections to the netconfd server are controlled through a main service loop, located in agt/agt_ncxserver.c.  It is expected that the 'select' loop in this file will be replaced in embedded systems.  The default netconfd server actually listens for local <ncx-connect> connections on an AF_LOCAL socket.  The openSSH server listens for connections on port 830 (or other configured TCP ports), and the netconf-subsystem thin client acts as a conduit between the SSH server and the netconfd server.

• Database management:  All configuration databases use a common configuration template, defined in ncx/cfg.h.  Locking and other generic database functions are handled in this module.  The actual manipulation of the value tree is handled by API functions in ncx/val.h, ncx/val_util.h,  agt/agt_val_parse.h, and agt/agt_val.h.

• NETCONF operations:  All standard NETCONF RPC callback functions are located in agt/agt_ncx.c.  All operations are completely automated, so there is no server instrumentation APIs in this file.

• NETCONF request processing:  All <rpc> requests and replies use common data structures and APIs, found in ncx/rpc.h and agt/agt_rpc.h.  Automated reply generation, automatic filter processing, and message state data is contained in the RPC message control block.

• NETCONF error reporting: All <rpc-error> elements use common data structures defined in ncx/rpc_err,h and agt/agt_rpcerr.h.  Most errors are handled automatically, but 'description statement' semantics need to be enforced by the SIL callback functions.  These functions use the API functions in agt/agt_util.h (such as agt_record_error) to generate data structures that will be translated to the proper <rpc-error> contents when a reply is sent.

• YANG module library management: All YANG modules are loaded into a common data structure (ncx_module_t) located in ncx/ncxtypes.h.   The API functions in ncx/ncxmod.h (such as ncxmod_load_module) are used to locate YANG modules, parse them, and store the internal data structures in a central library.  Multiple versions of the same module can be loaded at once, as required by YANG.

2.3.5 Session Control Block

Once a NETCONF session is started, it is assigned a session control block for the life of the session.  All NETCONF and system activity in driven through this interface, so the ncxserver loop can be replaced in an embedded system.

Each session control block (ses_scb_t) controls the input and output for one session, which is associated with one SSH user name.  Access control (see yuma-nacm.yang) is enforced within the context of a session control block.  Unauthorized return data is automatically removed from the response.  Unauthorized <rpc> or database write requests are automatically rejected with an 'access-denied' error-tag.

The user preferences for each session are also stored in this data structure.  They are initially derived from the server default values, but can be altered with the <set-my-session> operation and retrieved with the <get-my-session> operation.

2.3.6 Server Message Flows

The netconfd server provides the following type of components:

• NETCONF session management

• NETCONF/YANG database management

• NETCONF/YANG protocol operations

• Access control configuration and enforcement

• RPC error reporting

• Notification subscription management

• Default data retrieval processing

• Database editing

• Database validation

• Subtree and XPath retrieval filtering

• Dynamic and static capability management

• Conditional bbject management (if-feature, when)

• Memory management

• Logging management

• Timer services

All NETCONF and YANG protocol operation details are handled automatically within the netconfd server.  All database locking and editing is also handled by the server.  There are callback functions available at different points of the processing model for your module specific instrumentation code to process each server request, and/or generate notifications.  Everything except the 'description statement' semantics are usually handled

The server instrumentation stub files associated with the data model semantics are generated automatically with the yangdump program.  The developer fills in server callback functions to activate the networking device behavior represented by each YANG data model.

2.3.7 Main ncxserver Loop

The ncxserver loop does very little, and it is designed to be replaced in an embedded server that has its own SSH server:

• A client request to start an SSH session results in an SSH channel being established to an instance of the netconf-subsystem program.

• The netconf-subsystem program will open a local socket (/tmp/ncxserver.sock) and send a proprietary <ncxconnect> message to the netconfd server, which is listening on this local socket with a select loop (in agt_ncxserver.c).

• When a valid <ncxconnect> message is received by netconfd,  a new NETCONF session is created.

• After sending the <ncxconnect> message, the netconf-subsystem program goes into 'transfer mode', and simply passes input from the SSH channel to the netconfd server, and passes output from the netconfd server to the SSH server.

• The ncxserver loop simply waits for input on the open connections, with a quick timeout.  Each timeout, the server checks if a reboot, shutdown, signal, or other event occurred that needs attention.

• Notifications may also be sent during the timeout check, if any events are queued for processing.  The --max-burst configuration parameter controls the number of notifications sent to each notification subscription, during this timeout check.

• Input <rpc> messages are buffered, and when a complete message is received (based on the NETCONF End-of-Message marker), it is processed by the server and any instrumentation module callback functions that are affected by the request.

When the agt_ncxserver_run function in agt/agt_ncxserver.c is replaced within an embedded system, the replacement code must handle the following tasks:

• Call agt_ses_new_session in agt/agt_ses.c when a new NETCONF session starts.

• Call ses_accept_input in ncx/ses.c with the correct session control block when NETCONF data is received.

• Call agt_ses_process_first_ready in agt/agt_ses.c after input is received.  This should be called repeatedly until all serialized NETCONF messages have been processed.

• Call agt_ses_kill_session in agt/agt_ses.c when the NETCONF session is terminated.

• The following functions are used for sending NETCONF responses, if responses are buffered instead of sent directly (streamed).

  • ses_msg_send_buffs in ncx/ses_msg.c is used to output any queued send buffers.

• The following functions need to be called periodically:

  • agt_shutdown_requested in agt/agt_util.c to check if the server should terminate or reboot

  • agt_ses_check_timeouts in agt/agt_ses.c to check for idle sessions or sessions stuck waiting for a NETCONF <hello> message.

  • agt_timer_handler in agt/agt_timer.c to process server and SIL periodic callback functions.

  • send_some_notifications in agt/agt_ncxserver.c to process some outgoing notifications.

2.3.8 SIL Callback Functions

• Top Level: The top-level incoming messages are registered, not hard-wired, in the server message processing design.  The agt_ncxserver module accepts  the <ncxconnect> message from netconf-subsystem.  The agt_rpc module accepts the NETCONF <rpc> message.  Additional messages can be supported by the server using the top_register_node function.

• All RPC operations are implemented in a data-driven fashion by the server.  Each NETCONF operation is handled by a separate function in agt_ncx.c.  Any proprietary operation can be automatically supported, using the agt_rpc_register_method function.

  • Note:  Once the YANG module is loaded into the server, all RPC operations defined in the module are available.  If no SIL code is found, these will be dummy 'no-op' functions.  This mode can be used to provide some server simulation capability for client applications under development.

• All database operations are performed in a structured manner, using special database access callback functions.  Not all database nodes need callback functions.  One callback function can be used for each 'phase', or the same function can be used for multiple phases.  The agt_cb_register_callback function in agt/agt_cb.c is used by SIL code to hook into NETCONF database operations.

2.4 Server Operation

This section briefly describes the server internal behavior for some basic NETCONF operations.

2.4.1 Initialization

The file netconfd/netconfd.c contains the initial 'main' function that is used to start the server.

• The common services support for most core data structures is located in 'libncx.so'.  The 'ncx_init' function is called to setup these data structures.  This function also calls the bootstrap_cli function in ncx/ncx.c, which processes some key configuration parameters that need to be set right away, such as the logging parameters and the module search path.

• Most of the actual server code is located in the 'agt' directory.  The 'agt_init1' function is called to initialize core server functions.  The configuration parameters are processed, and the server profile is completed.

  • The agt_profile_t data structure in agt/agt.h is used to contain all the vendor-related boot-time options, such as the database target (candidate or running).  The init_server_profile function can be edited if the Yuma default values are not desired.  This will insure the proper factory defaults for server behavior are used, even if no configuration parameters are provided.

• The function init_server_profile in agt/agt.c is used to set the factory defaults for the server behavior.

The agt_init1 function also loads the core NETCONF protocol, netconfd CLI, and YANG data type modules.

• Note: netconfd uses yuma-netconf.yang, not ietf-netconf.yang to support a data-driven implementation.  The only difference is that the yuma version adds some data structures and extensions (such as ncx:root), to automate processing of all NETCONF messages.

After the core definition modules are loaded successfully, the agt_cli_process_input function in agt/agt_cli.c is called to process any command line and/or configuration file parameters that have been entered.

• Note: Any defaults set in the G module definitions will be added to the CLI parameter set.  The val_set_by_default function in ncx/val.c can be used to check if the node is set by the server to the YANG default value.  If not set, and the node has the YANG default value, then the client set this value explicitly.  This is different than the val_is_default function in ncx/val.c, which just checks if the node contains the YANG default value.

All the configuration parameters are saved, and those that can be processed right away are handled.  The agt_cli_get_valset function in agt/agt_cli.c can be used to retrieve the entire set of load-time configuration parameters.

2.4.2 Loading Modules and SIL Code

YANG modules and their associated device instrumentation can be loaded dynamically with the  --module configuration parameter.  Some examples are shown below:

module=foo

module=bar

module=baz@2009-01-05

module=~/mymodules/myfoo.yang

•  

• The ncxmod_find_sil_file function in ncx/ncxmod.c is used to find the library code associated with the each module name.  The following search sequence is followed:

  • Check the $YUMA_HOME/target/lib directory

  • Check each directory in the $YUMA_RUNPATH environment variable or --runpath configuration variable.

  • Check the /usr/lib/yuma directory

• If the module parameter contains any sub-directories or a file extension, then it is treated as a file, and the module search path will not be used.  Instead the absolute or relative file specification will be used.

• If the first term starts with an environment variable or the tilde (~) character, and will be expanded first

• If the 'at sign' (@) followed by a revision date is present, then that exact revision will be loaded.

• If no file extension or directories are specified, then the module search path is checked for YANG and YIN files that match.  The first match will be used, which may not be the newest, depending on the actual search path sequence.

• The $YUMA_MODPATH environment variable or --modpath configuration parameter can be used to configure one or more directory sub-trees to be searched.

• The $YUMA_HOME environment variable or --yuma-home configuration parameter can be used to specify the Yuma project tree to use if nothing is found in the currect directory or the module search path.

• The $YUMA_INSTALL environment variable or default Yuma install location (/usr/share/yuma/modules) will be used as a last resort to find a YANG or YIN file.

The server processes --module parameters by first checking if a dynamic library can be found which has an 'soname' that matches the module name.  If so, then the SIL phase 1 initialization function is called, and that function is expected to call the ncxmod_load_module function.

If no SIL file can be found for the module, then the server will load the YANG module anyway, and support database operations for the module, for provisioning purposes.  Any RPC operations defined in the module will also be accepted (depending on access control settings), but the action will not actually be performed.  Only the input parameters will be checked, and <or> or some <rpc-error> returned.

2.4.3 Core Module Initialization

The agt_init2 function in agt/agt.c is called after the configuration parameters have been collected.

• • Initialize the core server code modules

  • Static device-specific modules can be added to the agt_init2 function after the core modules have been initialized

  • Any 'module' parameters found in the CLI or server configuration file are processed.

  • The agt_cap_set_modules function in agt/agt_cap.c is called to set the initial module capabilities for the ietf-netconf-monitoring module

2.4.4 Startup Configuration Processing

After the static and dynamic server modules are loaded, the --startup (or --no-startup) parameter is processed by agt_init2 in agt/agt.c:

• If the --startup parameter is used and includes any sub-directories, it is treated as a file and must be found, as specified.

• Otherwise, the $YUMA_DATAPATH environment variable  or --datapath configuration parameter can be used to determine where to find the startup configuration file.

• If neither the --startup or --no-startup configuration parameter is present, then the data search path will be used to find the default startup-cfg.xml

• The $YUMA_HOME environment variable or --yuma-home configuration parameter is checked if no file is found in the data search path.  The $YUMA_HOME/data directory is checked if this parameter is set.

• The $YUMA_INSTALL environment variable or default location (/etc/yuma/) is checked next, if the startup configuration is still not found.

It is a fatal error if a startup config is specified and it cannot be found.

As the startup configuration is loaded, any SIL callbacks that have been registered will be invoked for the association data present in the startup configuration file..  The edit operation will be OP_EDITOP_LOAD during this callback.

After the startup configuration is loaded into the running configuration database, all the stage 2 initialization routines are called.  These are needed for modules which add read-only data nodes to the tree containing the running configuration.  SIL modules may also use their 'init2' function to create factory default configuration nodes (which can be saved for the next reboot).

2.4.5 Process an Incoming <rpc> Request

• PARSE Phase: The incoming buffer is converted to a stream of XML nodes, using the xmlTextReader functions from libxml2.  The agt_val_parse function is used to convert the stream of XML nodes to a val_value_t structure, representing the incoming request according to the YANG definition for the RPC operation.  An rpc_msg_t structure is also built for the request.

• VALIDATE Phase: If a message is parsed correctly, then the incoming message is validated according to the YANG machine-readable constraints.  Any description statement constraints need to be checked with a callback function.  The agt_rpc_register_method function in agt/agt_rpc.c is used to register callback functions.

• INVOKE Phase: If the message is validated correctly, then the invoke callback is executed.  This is usually the only required callback function.  Without it, the RPC operation has no affect.  This callback will set fields in the rpc_msg_t header that will allow the server to construct or stream the <rpc-reply> message back to the client.

• REPLY Phase: Unless some catastrophic error occurs, the server will generate an <rpc-reply> response.  If any <rpc-error> elements are needed, they are generated first.  If there is any response data to send, that is generated or streamed (via callback function provided earlier) at this time.  Any unauthorized data (according to to the yuma-nacm.yang module configuration) will be silently dropped from the message payload.  If there were no errors and no data to send, then an <ok> resonse is generated.

• POST_REPLY Phase: After the response has been sent, a rarely-used callback function can be invoked to cleanup any memory allocation or other data-model related tasks.  For example, if the rpc_user1 or rpc_user2 pointers in the message header contain allocated memory then they need to be freed at this time.

2.4.6 Edit the Database

• Validate Phase: The server will determine the edit operation and the actual nodes in the target database (candidate or running) that will be affected by the operation.  All of the machine-readable YANG statements which apply to the affected node(s) are tested against the incoming PDU and the target database.  If there are no errors, the server will search for a SIL validate callback function for the affected node(s).  If the SIL code has registered a database callback function for the node or its local ancestors, it will be invoked.  This SIL callback function usually checks additional constraints that are contained in the YANG description statements for the database objects.

• Test-Apply and Apply Phase: If the validate phase completes without errors, then the requested changes are applied to the target database.  If the target database is the running configuration, or if the edit-config 'test-option' parameter is set to 'test-then-set' (the default if --with-validate=true), then the test-apply phase is executed first.  This is essentially the same as the real apply phase, except that changes are made to a copy of the target database.  Once all objects have been altered as requested, the entire test database is validated, including all cross-referential integrity tests.  If this test completes without any errors, then the procedure is repeated on the real target database.

  • Note: This phase is used for the internal data tree manipulation and validation only.  It is not used to alter device behavior.  Resources may need to be reserved during the SIL apply callback, but the database changes are not activated at this time.

• Commit or Rollback Phase: If the validate and apply phases complete without errors, then then the server will search for SIL commit callback functions for the affected node(s) in the target database.  This SIL callback phase is used to apply the changes to the device and/or network.  It is only called when a commit procedure is attempted.  This can be due to a <commit> operation, or an <edit-config> or <copy-config> operation on the running database.

  • Note: If there are errors during the commit phase, then the backup configuration will be applied, and the server will search for a SIL callback to invoke with a 'rollback operation'.  The same procedure is used for confirmed commit operations which timeout or canceled by the client.

2.4.7 Save the Database

The following bullets describe how the server saves configuration changes to non-volatile storage:

• If the --with-startup=true parameter is used, then the server will support the :startup capability.  In this case, the <copy-config> command needs to be used to cause the running configuration to be saved.

• If the --with-startup=false parameter is used, then the server will not support the :startup capability.  In this case, the database will be saved each time the running configuration is changed.

• The <copy-config> or <commit> operations will cause the startup configuration file to be saved, even if nothing has changed.  This allows an operator to replace a corrupted or missing startup configuration file at any time.

• The database is saved with the agt_ncx_cfg_save function in agt/agt_ncx.c.

• The with-defaults 'explicit' mode is used during the save operation to filter the database contents.

  • Any values that have been set by the client will be saved in NV-storage.

  • Any value set by the server to a YANG default value will not be saved in the database.

  • If the server create a node that does not have a YANG default value (e.g., containers, lists, keys), then this node will be saved in NV storage.

• If the --startup=filespec parameter is used, then the server will save the database by overwriting that file.  The file will be renamed to backup-cfg.xml first.

• If the --no-startup parameter is used, or no startup file is specified and no default is found, then the server will create a file called 'startup-cfg.xml', in the following manner:

  • If the $YUMA_HOME variable is set, the configuration will be saved in $YUMA_HOME/data/startup-cfg.xml .

  • Otherwise, the configuration will be saved in $HOME/.yuma / startup-cfg.xml.

• The database is saved as an XML instance document, using the <config> element in the NETCONF 'base' namespace as the root element.  Each top-level YANG module supported by the server, which contains some explicit configuration data, will be saved as a child node of the <nc:config> element.  There is no particular order to the top-level data model elements.

2.5 Built-in Server Modules

There are several YANG modules which are implemented within the server, and not loaded at run-time like a dynamic SIL module.  Some of them are NETCONF standard modules and some are Yuma extension modules.

2.5.1 ietf-inet-types.yang

This module contains the standard YANG Internet address types.  These types are available for commonly used management object types.  A YANG module author should check this module first, before creating any new data types with the YANG typedef statement.

There are no accessible objects in this module, so there are no SIL callback functions.  The YANG data-types are supported within the Yuma engine core modules, such as ncx/val.c and ncx/xml_wr.c.

2.5.2 ietf-netconf-monitoring.yang

The standard NETCONF Monitoring module is used to examine the capabilities, current state, and statistics related to the NETCONF server.  The entire module is supported.

This module is also used to retrieve the actual YANG or YIN files (or URLs for them) that the server is using.  Clients can use the <get-schema> RPC operation to retrieve the YANG or YIN files listed in the /netconf-state/schemas subtree.  A client will normally check the <hello> message from the server for module capabilities, and use its own local copy of a server YANG module, if it can.  If not, then the <get-schema> function can be used to retrieve the YANG module.

The agt/agt_state.c contains the SIL callback functions for this module.

2.5.3 ietf-with-defaults.yang

The standard <with-defaults> extension to some NETCONF operations is defined in this module.  This parameter is added to the <get>, <get-config>, and <copy-config> operations to let the client control how 'default leafs' are returned by the server.  The Yuma server can be configured to use any of the default handling styles (report-all, trim, or explicit).  The filtering of default nodes is handled automatically by the server support functions in agt/agt_util.c, and the XML write functions in ncx/xml_wr.c.

2.5.4 ietf-yang-types.yang

This module contains the standard YANG general user data types.  These types are available for commonly used derived types.  A YANG module author should check this module first, before creating any new data types with the YANG typedef statement.

There are no accessible objects in this module, so there are no SIL callback functions.  The YANG data-types are supported within the Yuma engine core modules, such as ncx/val.c and ncx/xml_wr.c.

2.5.5 nc-notifications.yang

This module is defined in RFC 5277, the NETCONF Notifications specification.  It contains the <replayComplete> and <notificationComplete> notification event definitions.

The file agt/agt_not.c contains the SIL support code for this module.

2.5.6 notifications.yang

This module is defined in RFC 5277, the NETCONF Notifications specification.  All of this RFC is supported in the server.  This module contains the <create-subscription> RPC operation.  The notification replay feature is controlled with the --eventlog-size configuration parameter.  The <create-subscription> operation is fully supported, including XPath and subtree filters.  The yuma-nacm module can be used to control what notification events a user is allowed to receive.  The <create-subscription> filter allows the client to select which notification events it wants to receive.

The file agt/agt_not.c contains the SIL callback functions for this modules.

2.5.7 yuma-app-common.yang

This module contains some common groupings of CLI parameters supported by some or all Yuma programs.  Each program with CLI parameters defines its own module of CLI parameters (using the ncx:cli extension).  The program name is used for the YANG module name as well (e.g., yangdump.yang or netconfd.yang).

The SIL callback functions for the common groupings in this module are found in ncx/val_util.c, such as the val_set_feature_parms function.

2.5.8 yuma-interfaces.yang

This module contains the Yuma interfaces table, which is just a skeleton configuration list, plus some basic interface counters.  This module is intended to provide an example for embedded developers to replace this module with their own interfaces table.  The Yuma table uses information in some files found in Unix systems which support the /proc/net/dev system file.

The file agt/agt_if.c contains the SIL callback functions for this module.

2.5.9 yuma-mysession.yang

This module provides the Yuma proprietary <get-my-session> and <set-my-session> RPC operations.  These are used by the client to set some session output preferences, such as the desired line length, indentation amount, and defaults handling behavior.

The file agt/agt_ses.c contains the SIL callback functions for this module.

2.5.10 yuma-nacm.yang

This module contains the Yuma NETCONF Access Control Model implementation.  It provides all user-configurable access control settings and also provides API functions to check if a specific access request should be allowed or not.

The file agt/agt_acm.c contains the SIL callback functions for this module.

2.5.11 yuma-ncx.yang

This module provides the YANG language extension statements that are used by Yuma programs to automate certain parts of the NETCONF protocol, document generation, code generation, etc.

There are no SIL callback functions for this module.  There are support functions within the src/ncx directory that include the obj_set_ncx_flags function in ncx/obj.c

2.5.12 yuma-netconf.yang

The NETCONF protocol operations, message structures, and error information are all data-driven, based on the YANG statements in the yuma-netconf.yang module.  The ietf-netconf.yang module is not used at this time because it does not contain the complete set of YANG statements needed.  The yuma-netconf.yang version is a super-set of the IETF version.  Only one YANG module can be associated with an XML namespace in Yuma.  In a future version, the extra data structures will be moved to an annotation module.

The file agt/agt_ncx.c contains the SIL callback functions for this module.

This module is not advertised in the server capabilities.  It is  only used internally within the server.

2.5.13 yuma-proc.yang

This module provides some Unix /proc file-system data, in nested XML format.  This module will not load if the files /proc/meminfo and /proc/cpuinfo are not found.

The file agt/agt_proc.c contains the SIL callback functions for this module.

2.5.14 yuma-system.yang

This module contains the Yums /system data structure, providing basic server information, unix 'uname' data, and all the Yuma proprietary notification event definitions.

The file agt/agt_sys.c contains the SIL callback functions for this module.

2.5.15 yuma-types.yang

This module provides some common data types that are used by other Yuma YANG modules.

There are no SIL callback functions for this module.

3 YANG Objects and Data Nodes

This section describes the basic design of the YANG object tree and the corresponding data tree that represents instances of various object nodes that the client or the server can create.

3.1 Object Definition Tree

The object tree is a tree representation of all the YANG module rpc, data definition, and notification statements.  It starts with a 'root' container.  This is defined with a YANG container statement which has an ncx:root extension statement within it.  The <config> parameter within the <edit-config> operation is an example of an object node which is treated as a root container.  Each configuration database maintained by the server (e.g., <candidate> and <running>) has a root container value node as its top-level object.

A root container does not have any child nodes defined in it within the YANG file.  However, the Yuma tools will treat this special container as if any top-level YANG data node is allowed to be a child node of the 'root' container type.

3.1.1 Object Node Types

There are 14 different YANG object node types, and a discriminated union of sub-data structures contains fields common to each sub-type. Object templates are defined in ncx/obj.h.

YANG Object Types

3.1.2 Object Node Template (obj_template_t)

The following typedef is used to represent an object tree node:

/* One YANG data-def-stmt */

typedef struct obj_template_t_ {

    dlq_hdr_t      qhdr;

    obj_type_t     objtype;

    uint32         flags;              /* see OBJ_FL_* definitions */

    ncx_error_t    tkerr;

    grp_template_t *grp;          /* non-NULL == in a grp.datadefQ */

    /* 4 back pointers */

    struct obj_template_t_ *parent;

    struct obj_template_t_ *usesobj;

    struct obj_template_t_ *augobj;

    struct xpath_pcb_t_    *when;           /* optional when clause */

    dlq_hdr_t               metadataQ;       /* Q of obj_metadata_t */

    dlq_hdr_t               appinfoQ;         /* Q of ncx_appinfo_t */

    dlq_hdr_t               iffeatureQ;     /* Q of ncx_iffeature_t */

    /* cbset is agt_rpc_cbset_t for RPC or agt_cb_fnset_t for OBJ */

    void                   *cbset;   

    /* object namespace ID assigned at runtime

     * this can be changed over and over as a

     * uses statement is expanded.  The final

     * expansion into a real object will leave

     * the correct value in place

     */

    xmlns_id_t             nsid;

    union def_ {

obj_container_t   *container;

obj_leaf_t        *leaf;

obj_leaflist_t    *leaflist;

obj_list_t        *list;

obj_choice_t      *choic;

obj_case_t        *cas;

obj_uses_t        *uses;

obj_refine_t      *refine;

obj_augment_t     *augment;

obj_rpc_t         *rpc;

obj_rpcio_t       *rpcio;

obj_notif_t       *notif;

    } def;

} obj_template_t;

The following table highlights the fields within the obj_template_t data structure:

obj_template_t Fields

3.1.3 obj_template_t Access Functions

The file ncx/obj.h contains many API functions so that object properties do not have to be accessed directly.  The following table highlights the most commonly used functions.  Refer to the H file for a complete definition of each API function.

obj_template_t Access Functions

3.2 Data Tree

A Yuma data tree is a representation of some subset of all possible object instances that a server is maintaining within a configuration database or other structure.

Each data tree starts with a 'root' container, and any child nodes represent top-level YANG module data nodes that exist within the server.

Each configuration database maintains its own copy (and version) of the data tree.  There is only one object tree, however, and all data trees use the same object tree for reference.

Not all object types have a corresponding node within a data tree.  Only 'real' data nodes are present.  Object nodes that are used as meta-data to organize the object tree (e.g., choice, augment) are not present.  The following table lists the object types and whether each one is found in a data tree.

Object Types in the Data Tree

3.2.1 Data Node Types

The ncx_btype_t enumeration in ncx/ncxtypes.h is used within each val_value_t to quickly identify which variant of the data node structure is being used.

The following table describes the different enumeration values:

Yuma Data Types (ncx_btype_t)

3.2.2 Yuma Data Node Edit Variables (val_editvars_t)

There is a temporary data structure which is attached to a data node while editing operations are in progress, called val_editvars_t.  This structure is used by the functions in agt/agt_val.c to manipulate the value tree nodes during an <edit-config>, <copy-config>, <load-config>, or <commit> operation.

The SIL callback functions may wish to refer to the fields in this data structure.  There is also a SIL cookie field to allow data to be transferred from one callback stage to the later stages.  For example, if an edit operation caused the device instrumentation to reserve some memory, then this cookie could store that pointer.

The following typedef is used to define the val_editvars_t structure:

/* one set of edit-in-progress variables for one value node */

typedef struct val_editvars_t_ {

    /* these fields are only used in modified values before they are

     * actually added to the config database (TBD: move into struct)

     * curparent == parent of curnode for merge

     */

    struct val_value_t_  *curparent;      

    op_editop_t    editop;            /* effective edit operation */

    op_insertop_t  insertop;             /* YANG insert operation */

    xmlChar       *insertstr;          /* saved value or key attr */

    struct xpath_pcb_t_ *insertxpcb;       /* key attr for insert */

    struct val_value_t_ *insertval;                   /* back-ptr */

    boolean        iskey;                     /* T: key, F: value */

    boolean        operset;                  /* nc:operation here */

    void          *pcookie;                /* user pointer cookie */

    int            icookie;                /* user integer cookie */

} val_editvars_t;

The following fields within the val_editvars_t are highlighted:

val_editvars_t Fields

3.2.3 Yuma Data Nodes (val_value_t)

The val_value_t data structure is used to maintain the internal representation of all NETCONF databases, non-configuration data available with the <get> operation, all RPC operation input and output parameters, and all notification contents.

The following typedef is used to define a value node:

/* one value to match one type */

typedef struct val_value_t_ {

    dlq_hdr_t      qhdr;

    /* common fields */

    struct obj_template_t_ *obj;        /* bptr to object def */

    typ_def_t *typdef;              /* bptr to typdef if leaf */

    const xmlChar   *name;                /* back pointer to elname */

    xmlChar         *dname;          /* AND malloced name if needed */

    struct val_value_t_ *parent;       /* back-ptr to parent if any */

    xmlns_id_t     nsid;              /* namespace ID for this node */

    ncx_btype_t    btyp;                 /* base type of this value */

    uint32         flags;                  /* internal status flags */

    ncx_data_class_t dataclass;             /* config or state data */

    /* YANG does not support user-defined meta-data but NCX does.

     * The <edit-config>, <get> and <get-config> operations

     * use attributes in the RPC parameters, the metaQ is still used

     *

     * The ncx:metadata extension allows optional attributes

     * to be added to object nodes for anyxml, leaf, leaf-list,

     * list, and container nodes.  The config property will

     * be inherited from the object that contains the metadata

     *

     * This is used mostly for RPC input parameters

     * and is strongly discouraged.  Full edit-config

     * support is not provided for metdata

     */

    dlq_hdr_t        metaQ;                      /* Q of val_value_t */

    /* value editing variables */

    val_editvars_t  *editvars;              /* edit-in-progress vars */

    status_t         res;                      /* validationt result */

    /* Used by Agent only:

     * if this field is non-NULL, then the entire value node

     * is actually a placeholder for a dynamic read-only object

     * and all read access is done via this callback function;

     * the real data type is getcb_fn_t *

     */

    void *getcb;

    /* if this field is non-NULL, then a malloced value struct

     * representing the real value retrieved by

     * val_get_virtual_value, is cached here for XPath filtering

     * TBD: add timestamp to reuse cached entries for some time

     * period

     */

    struct val_value_t_ *virtualval;

    /* these fields are used for NCX_BT_LIST */

    struct val_index_t_ *index;   /* back-ptr/flag in use as index */

    dlq_hdr_t       indexQ;    /* Q of val_index_t or ncx_filptr_t */

    /* this field is used for NCX_BT_CHOICE

     * If set, the object path for this node is really:

     *    $this --> casobj --> casobj.parent --> $this.parent

     * the OBJ_TYP_CASE and OBJ_TYP_CHOICE nodes are skipped

     * inside an XML instance document

     */

    struct obj_template_t_   *casobj;

    /* these fields are for NCX_BT_LEAFREF

     * NCX_BT_INSTANCE_ID, or tagged ncx:xpath

     * value stored in v union as a string

     */

    struct xpath_pcb_t_            *xpathpcb;

    /* union of all the NCX-specific sub-types

     * note that the following invisible constructs should

     * never show up in this struct:

     *     NCX_BT_CHOICE

     *     NCX_BT_CASE

     *     NCX_BT_UNION

     */

    union v_ {

/* complex types have a Q of val_value_t representing

* the child nodes with values

*   NCX_BT_CONTAINER

*   NCX_BT_LIST

*/

        dlq_hdr_t   childQ;         

        /* Numeric data types:

*   NCX_BT_INT8, NCX_BT_INT16,

*   NCX_BT_INT32, NCX_BT_INT64

*   NCX_BT_UINT8, NCX_BT_UINT16

*   NCX_BT_UINT32, NCX_BT_UINT64

*   NCX_BT_DECIMAL64, NCX_BT_FLOAT64

*/

ncx_num_t   num;

/* String data types:

*   NCX_BT_STRING

*   NCX_BT_INSTANCE_ID

*/

ncx_str_t  str;

val_idref_t idref;

ncx_binary_t binary;              /* NCX_BT_BINARY */

ncx_list_t list;      /* NCX_BT_BITS, NCX_BT_SLIST */

boolean    boo;    /* NCX_BT_EMPTY, NCX_BT_BOOLEAN */

ncx_enum_t enu;       /* NCX_BT_UNION, NCX_BT_ENUM */

xmlChar   *fname;                 /* NCX_BT_EXTERN */

xmlChar   *intbuff;               /* NCX_BT_INTERN */

    } v;

} val_value_t;

The following table highlights the fields in this data structure:

val_value_t Fields

3.2.4 val_value_t Access Macros

There are a set of macros defined to access the fields within a val_value_t structure.

These should be used instead of accessing the fields directly.  There are also functions defined as well.   These macros are provided in addition the the access functions for quick access to the actual node value.  These macros must only be used when the base type ('btyp') field has been properly set and known by the SIL code.  Some auto-generated SIL code uses these macros.

The following table summarized the val_value_t macros that are defined in ncx/val.h:

3.2.5 val_value_t Access Functions

The file ncx/val.h contains many API functions so that object properties do not have to be accessed directly.  In addition, the file ncx/val_util.h contains more (high-level) utility functions.  The following table highlights the most commonly used functions.  Refer to the H files for a complete definition of each API function.

val_value_t Access Functions

3.2.6 SIL Utility Functions

There are some high-level SIL callback utilities in agt/agt_util.h.  These functions access the lower-level functions in libncx to provide simpler functions for common SIL tasks.

The following table highlights the functions available in this module:

agt/agt_util Functions

4 SIL External Interface

Each SIL has 2 initialization functions and 1 cleanup function that must be present.

• The first initialization callback function is used to set up the configuration related objects.

• The second initialization callback is used to setup up non-configuration objects, after the running configuration has been loaded from the startup file.

• The cleanup callback is used to remove all SIL data structures and unregister all callback functions.

These are the only SIL functions that the server will invoke directly.  They are generated by yangdump  with the --format=c parameter, and usually do not require any editing by the developer.

Most of the work done by SIL code is through callback functions for specific RPC operations and database objects.  These callback functions are registered during the initialization functions.

4.1 Stage 1 Initialization

The stage 1 initialization function is the first function called in the library by the server.

If the netconfd configuration parameters include a 'load' command for the module, then this function will be called during server initialization.  It can also be called if the <load> operation is invoked during server operation.

This function MUST NOT attempt to access any database.  There will not be any configuration databases if this function is called during server initialization.  Use the 'init2' function to adjust the running configuration.

This callback function is expected to perform the following functions:

• initialize any module static data

• make sure the requested module name and optional revision date parameters are correct

• load the requested module name and revision with ncxmod_load_module

• setup top-level object cache pointers (if needed)

• register any RPC method callbacks with agt_rpc_register_method

• register any database object callbacks with agt_cb_register_callback

• perform any device-specific and/or module-specific initialization

Name Format:

y_<modname>_init

Input:

• modname == string containing module name to load

• revision == string containing revision date to use
               == NULL if the operator did not specify a revision.

Returns:

• operation status (0 if success)

Example function generated by yangdump:

/********************************************************************

* FUNCTION y_toaster_init

*

* initialize the toaster server instrumentation library

*

* INPUTS:

*    modname == requested module name

*    revision == requested version (NULL for any)

*

* RETURNS:

*     error status

********************************************************************/

status_t

    y_toaster_init (

        const xmlChar *modname,

        const xmlChar *revision)

{

    agt_profile_t *agt_profile;

    status_t res;

    y_toaster_init_static_vars();

    /* change if custom handling done */

    if (xml_strcmp(modname, y_toaster_M_toaster)) {

        return ERR_NCX_UNKNOWN_MODULE;

    }

    if (revision && xml_strcmp(revision, y_toaster_R_toaster)) {

        return ERR_NCX_WRONG_VERSION;

    }

    agt_profile = agt_get_profile();

    res = ncxmod_load_module(

        y_toaster_M_toaster,

        y_toaster_R_toaster,

        &agt_profile->agt_savedevQ,

        &toaster_mod);

    if (res != NO_ERR) {

        return res;

    }

    toaster_obj = ncx_find_object(

        toaster_mod,

        y_toaster_N_toaster);

    if (toaster_mod == NULL) {

        return SET_ERROR(ERR_NCX_DEF_NOT_FOUND);

    }

    make_toast_obj = ncx_find_object(

        toaster_mod,

        y_toaster_N_make_toast);

    if (toaster_mod == NULL) {

        return SET_ERROR(ERR_NCX_DEF_NOT_FOUND);

    }

    cancel_toast_obj = ncx_find_object(

        toaster_mod,

        y_toaster_N_cancel_toast);

    if (toaster_mod == NULL) {

        return SET_ERROR(ERR_NCX_DEF_NOT_FOUND);

    }

    toastDone_obj = ncx_find_object(

        toaster_mod,

        y_toaster_N_toastDone);

    if (toaster_mod == NULL) {

        return SET_ERROR(ERR_NCX_DEF_NOT_FOUND);

    }

    res = agt_rpc_register_method(

        y_toaster_M_toaster,

        y_toaster_N_make_toast,

        AGT_RPC_PH_VALIDATE,

        y_toaster_make_toast_validate);

    if (res != NO_ERR) {

        return res;

    }

    res = agt_rpc_register_method(

        y_toaster_M_toaster,

        y_toaster_N_make_toast,

        AGT_RPC_PH_INVOKE,

        y_toaster_make_toast_invoke);

    if (res != NO_ERR) {

        return res;

    }

    res = agt_rpc_register_method(

        y_toaster_M_toaster,

        y_toaster_N_cancel_toast,

        AGT_RPC_PH_VALIDATE,

        y_toaster_cancel_toast_validate);

    if (res != NO_ERR) {

        return res;

    }

    res = agt_rpc_register_method(

        y_toaster_M_toaster,

        y_toaster_N_cancel_toast,

        AGT_RPC_PH_INVOKE,

        y_toaster_cancel_toast_invoke);

    if (res != NO_ERR) {

        return res;

    }

    res = agt_cb_register_callback(

        y_toaster_M_toaster,

        (const xmlChar *)"/toaster",

        (const xmlChar *)"2009-11-20",

        y_toaster_toaster_edit);

    if (res != NO_ERR) {

        return res;

    }

    /* put your module initialization code here */

    return res;

} /* y_toaster_init */

4.2 Stage 2 Initialization

The stage 2 initialization function is the second function called in the library by the server:

• It will only be called if the stage 1 initialization is called first, and it returns 0 (NO_ERR status).

• This function is used to initialize any needed data structures in the running configuration, such as factory default configuration, read-only counters and status objects.

• It is called after the startup configuration has been loaded into the server.

• If the <load> operation is used during server operation, then this function will be called immediately after the state 1 initialization function.

Note that configuration data structures that are loaded during server initialization (load_running_config) will be handled by the database callback functions registered during phase 1 initialization.

Any server-created configuration nodes should be created during phase 2 initialization (this function), after examining the explicitly-provided configuration data.  For example, the top-level /nacm container will be created (by agt_acm.c) if it is not provided in the startup configuration.

This callback function is expected to perform the following functions:

• load non-configuration data structures into the server (if needed)

• initialize top-level data node cache pointers (if needed)

• load factory-default configuration data structures into the server (if needed)

• optionally save a cached pointer to a data tree node (such as the root node for the module).  The agt_create_cache function in agt/agt_util.h is used to initialize such a module-static variable.

Name Format:

y_<modname>_init2

Returns:

• operation status (0 if success)

Example function generated by yangdump:

/********************************************************************

* FUNCTION y_toaster_init2

*

* SIL init phase 2: non-config data structures

* Called after running config is loaded

*

* RETURNS:

*     error status

********************************************************************/

status_t

    y_toaster_init2 (void)

{

    status_t res;

    res = NO_ERR;

    toaster_val = agt_init_cache(

        y_toaster_M_toaster,

        y_toaster_N_toaster,

        &res);

    if (res != NO_ERR) {

        return res;

    }

    /* put your init2 code here */

    return res;

} /* y_toaster_init2 */

4.3 Cleanup

The cleanup function is called during server shutdown.  It is only called if the stage 1 initialization function is called.  It will be called right away if either the stage 1 or stage 2 initialization functions return a non-zero error status.

This callback function is expected to perform the following functions:

• cleanup any module static data

• free any top-level object cache pointers (if needed)

• unregister any RPC method callbacks with agt_rpc_unregister_method

• unregister any database object callbacks with agt_cb_unregister_callbacks

• perform any device-specific and/or module-specific cleanup

Name Format:

y_<modname>_cleanup

Example function generated by yangdump:

/********************************************************************

* FUNCTION y_toaster_cleanup

*    cleanup the server instrumentation library

*

********************************************************************/

void

    y_toaster_cleanup (void)

{

    agt_rpc_unregister_method(

        y_toaster_M_toaster,

        y_toaster_N_make_toast);

    agt_rpc_unregister_method(

        y_toaster_M_toaster,

        y_toaster_N_cancel_toast);

    agt_cb_unregister_callbacks(

        y_toaster_M_toaster,

        (const xmlChar *)"/toaster");

    /* put your cleanup code here */

} /* y_toaster_cleanup */

5 SIL Callback Interface

This section briefly describes the SIL code that a developer will need to create to handle the data-model specific details.  SIL functions access internal server data structures, either directly or through utility functions.  Database mechanics and XML processing are done by the server engine, not the SIL code.  A more complete reference can be found in section 5.

When a <rpc> request is received, the NETCONF server engine will perform the following tasks before calling any SIL:

• parse the RPC operation element, and find its associated YANG rpc template

• if found, check if the session is allowed to invoke this RPC operation

• if the RPC is allowed, parse the rest of the XML message, using the rpc_template_t for the RPC operation to determine if the basic structure is valid.

• if the basic structure is valid, construct an rpc_msg_t data structure for the incoming message.

• check all YANG machine-readable constraints, such as must, when, if-feature, min-elements, etc.

• if the incoming message is completely 'YANG valid', then the server will check for an RPC validate function, and call it if found.  This SIL code is only needed if there are additional system constraints to check.  For example:

  • need to check if a configuration name such as <candidate/> is supported

  • need to check if a configuration database is locked by another session

  • need to check description statement constraints not covered by machine-readable constraints

  • need to check if a specific capability or feature is enabled

• If the validate function returns a NO_ERR status value, then the server will call the SIL invoke callback, if it is present.  This SIL code should always be present, otherwise the RPC operation will have no real affect on the system.

• At this point, an <rpc-reply> is generated, based on the data in the rpc_msg_t.

  • Errors are recorded in a queue when they are detected.

  • The server will handle the error reply generation for all errors it detects.

  • For SIL detected errors, the agt_record_error function in agt/agt_util.h is usually used to save the error details.

  • Reply data can be generated by the SIL invoke callback function and stored in the rpc_msg_t structure.

  • Replay data can be streamed by the SIL code via reply callback functions.  For example, the <get> and <get-config> operations use callback functions to deal with filters, and stream the reply by walking the target data tree.

• After the <rpc-reply> is sent, the server will check for an RPC post reply callback function.  This is only needed if the SIL code allocated some per-message data structures.  For example, the rpc_msg_t contains 2 SIL controlled pointers (rpc_user1 and rpc_user2).  The post reply callback is used by the SIL code to free these pointers, if needed.

The database edit SIL callbacks are only used for database operations that alter the database.  The validate and invoke callback functions for these operations will in turn invoke the data-model specific SIL callback functions, depending on the success or failure of the edit request.

5.1 RPC Operation Interface

All RPC operations are data-driven within the server, using the YANG rpc statement for the operation and SIL callback functions.

Any new protocol operation can be added by defining a new YANG rpc statement in a module, and providing the proper SIL code.

5.1.1 RPC Callback Initialization

The agt_rpc_register_method function in agt/agt_rpc.h is used to provide a callback function for a specific callback phase.  The same function can be used for multiple phases if desired.

/* Template for RPC server callbacks

 * The same template is used for all RPC callback phases

 */

typedef status_t

    (*agt_rpc_method_t) (ses_cb_t *scb,

rpc_msg_t *msg,

xml_node_t *methnode);

extern status_t

    agt_rpc_register_method (const xmlChar *module,

                             const xmlChar *method_name,

                             agt_rpc_phase_t  phase,

                             agt_rpc_method_t method);

agt_rpc_register_method

5.1.2 RPC Message Header

The NETCONF server will parse the incoming XML message and construct an RPC message header, which is used to maintain state and any other message-specific data during the processing of an incoming <rpc> request.

The rpc_msg_t data structure in ncx/rpc.h is used for this purpose.  The following table summarizes the fields:

rpc_msg_t