Custom Code Router

The Custom Code Router allows you to extend Marshals harvesting functionality with your own source code or a custom assembly. If there's no ready-to-use harvester for your data source or if you need detailed control over the harvesting process itself, this is the solution to use.

To use your custom harvester:

In Visual Studio, implement a .net harvester class compatible with the Custom Router. See specification and boilerplate examples below.
Place your source code, or assembly, in the code directory of the Custom Router. This folder is specified in the Marshal.Router.Custom.exe.config file found in the installation directory.
If not running, start the Marshal Custom Router Windows service.
Within your Marshal model, create a TCP JSON Harvester, point it to the Custom Router and configure it to use your code.

The Custom Router application is a Windows service. To start and stop the service open the Server Manager application and click on Configuration then followed by Services. If installed, you'll find the Marshal Custom Router in the services list. Errors and warnings occuring during execution are reported to the Event Viewer application log on the computer.

Configuring the JSON TCP Harvester

Harvester
Name	TCP JSON Harvester
Router
Connection String	This string is mandatory and passed to the custom code. Use it, if applicable, to connect to a remote system or similar.
For name	This determines what code to load, compile and run. It's a comma separated list according to: Assembly \| File, Namespace.Class [, Referenced assembly, ... , Referenced assembly] where the first part is either your compiled custom code assembly or a single c# source code file, to be compiled by the Custom Router. Note, your assembly file, or source code, must be placed in specific folder on the computer. This folder is specified in the Custom Router config-file. The second part, Namespace.Class, identifies the class implementing your custom harvester, ie it's the class to run! Finally, you can provide a list of referenced assemblies. For instance, to load a driver for a remote system.
Host	IP Address of the Custom Router, e.g. localhost or 127.0.0.1.
Port	Port on which the Custom Router Windows service listens for connections from the harvester. By default, this is port 8083.
Settings
Group by	Not used
Order by	Not used
Query Timeout	Not used
Table or View	This string is passed to the custom code. Use it in your code to determine what table, or type of object, to harvest from.
Where	Used to manually pass arguments to the harvester. These arguments are added to the set of arguments assigned by the parent query node, see Query node Relation property. Enter arguments in SQL Syntax separated by an AND, eg ReadOnly=1 and Extension='.txt'. It's entirely up to the custom harvester handle these arguments!
User authentication
Password	This string is passed to the custom code, use it if applicable.
User ID	This string is passed to the custom code, use it if applicable.

Implementing a harvester class

A Custom Router harvester is a plain c# class implementing IDisposable and a set of methods to support browsing and harvesting your data source. You don't need any SDK to do the development, ie your code doesn't depend on Marshal and should be developed and tested entirely on its own merits.

You could download our Animal sample and use it as a boilerplate for your solution. It's a minimal sample class, containing some embedded data, that illustrates most aspects. The download also contains a sample Marshal model. Don't forget to point the Custom Router to folder where you've place the code. Or you may start from the code snippet below.

public class YourClass : IDisposable

{

// Browser method

public Dictionary<string, object> Database() { return new Dictionary<string,object>(); }

public List<string> Tables() { return new List<string>(); }

public List<string> Columns(string table) { return new List<string>(); }

public Dictionary<string, object> Column(string table, string column) { return new Dictionary<string,object>(); }

// Query methods

public void Query(string table, List<string> columns, Dictionary<string, string> parameters) { }

public bool Read() { return false; }

public List<object> GetValues() { return new List<object>(); }

public YourClass(string connection, string userID, string password) { }

public void Dispose() { }

}

The object constuctor receives three input arguments: connection, userID and password. These correspond to the Configuration string, User ID and Password properties of the harvester configuration. Your class needs to implement the IDisposable interface. Free any external resources, such as connection and file handles, in the Dispose method.

The rest of the methods can be diveded into browsing methods, that describe the data source, and query methods for retrieving data from the source.

Browser methods
Database	Returns a dictionary of named properties describing the data source as such. The supported properties are database_product_name, database_major_verson, database_minor_version.
Tables	Returns a list of tables (object names) exposed by the data source.
Columns	Returns the list of available columns (object properties) for a specific table.
Column	Returns a dictionary of named properties describing a specific table column. The following properties are supported: data_type is a mandatory integer representing the data type of the column. These integers corresponds to the java.sql.Types constants, see list below. column_size is an integer representing the size of the column. This property is optional. Set it if the column is a limited size string or a decimal with fixed size. decimal_digits is an integer representing the number of digits following the decimal point. This property is optional. Set this if the column is an exact decimal. nullable is an integer marking the column as nullable. Set this to 1 (one) if the column is nullable.
Query methods
Query	The query method accepts three arguments: The name of the table from which to harvest data. This the value entered into the Table or View property of the harvester configuration. The columns to be returned from the table. In the Marshal model, this is the list of leafs of the query node being harvested. A dictionary of name/value-pairs. Use these to filter what data, ie what rows, to return from the table. This dictionary is the union of the Where property and relation to parent of the query node. This method doesn't return anything and it's entirely up to your implementation whether to execute any code or not at this point. Your code should, however, generate an exception if the input doesn't met the requirements. This exception message will be displayed in the Marshal Editor or, in the case of backend execution, logged to the server Event viewer.
Read	This method should return true if more data can be retrieved using GetValues and false otherwise. The method is used in the following manner: Query(...) while Read() GetValues(...) Typically, this method would read a set of values from the underlying data store and advance the read cursor.
GetValues	This method returns a list of data, in the same order as the columns passed to the Query method, for the row last read. For each column, the data type of the returned value should be consistent with the data_type property returned by the Column method.

Common SQL Data types
java.sql.Types.LONGNVARCHAR	-16
java.sql.Types.NCHAR	-15
java.sql.Types.NVARCHAR	-9
java.sql.Types.BIT	-7
java.sql.Types.TINYINT	-6
java.sql.Types.BIGINT	-5
java.sql.Types.LONGVARBINARY	-4
java.sql.Types.VARBINARY	-3
java.sql.Types.BINARY	-2
java.sql.Types.LONGVARCHAR	-1
java.sql.Types.NULL	0
java.sql.Types.CHAR	1
java.sql.Types.NUMERIC	2
java.sql.Types.DECIMAL	3
java.sql.Types.INTEGER	4
java.sql.Types.SMALLINT	5
java.sql.Types.FLOAT	6
java.sql.Types.REAL	7
java.sql.Types.DOUBLE	8
java.sql.Types.VARCHAR	12
java.sql.Types.BOOLEAN	16
java.sql.Types.DATE	91
java.sql.Types.TIME	92
java.sql.Types.TIMESTAMP	93