Introducing WSDL Integration Module in ClaySys AppForms

ClaySys AppForms Version 1.0



Introduction to WSDL ( Web Service Definition Language )

WSDL is an XML format for describing network services as a set of endpoints operating on messages containing either document-oriented or procedure-oriented information. The operations and messages are described abstractly, and then bound to a concrete network protocol and message format to define an endpoint. Related concrete endpoints are combined into abstract endpoints (services). WSDL is extensible to allow description of endpoints and their messages regardless of what message formats or network protocols are used to communicate, however, the only bindings described in this document describe how to use WSDL in conjunction with SOAP 1.1, HTTP GET/POST, and MIME.

A WSDL document defines services as collections of network endpoints, or ports. In WSDL, the abstract definition of endpoints and messages is separated from their concrete network deployment or data format bindings. This allows the reuse of abstract definitions: messages, which are abstract descriptions of the data being exchanged, and port types which are abstract collections of operations. The concrete protocol and data format specification for a particular port type constitutes a reusable binding. A port is defined by associating a network address with a reusable binding, and a collection of ports define a service. Hence, a WSDL document uses the following elements in the definition of network services:

Types– a container for data type definitions using some type system (such as XSD).

Message– an abstract, typed definition of the data being communicated.

Operation– an abstract description of an action supported by the service.

Port Type–an abstract set of operations supported by one or more endpoints.

Binding– a concrete protocol and data format specification for a particular port type.

Port– a single endpoint defined as a combination of a binding and a network address.

Service– a collection of related endpoints.


Configuring the WSDL and Reading the available web methods

    1. The basic requirement will be that the ClaySys AppForms application should be able to access any external web service and allow the application to make calls to it.
    1. When a new WSDL is introduced to the application, the end user will be adding the URI of the WSDL to the application along with the Credentials if any.
    1. Once the URI and credentials are provided, the application will display all the web methods available from the WSDL.
    1. When the user selects one web method, the application will display the parameters that need to be passed to this web service and the return type of the web service.
    2. The WSDL Configuration steps are explained below

      NewItem10
      Create a new DataSource

      Configure
      Configure a DataSource Step1

       

      Select
      Select DataSource

       

      Display
      Display available Web Methods

       

      Display
      Display Parameters for the selected Web Method

       

      In case of a XML parameter the administrator will be able to define the XML parameter structure. If there is a XML file that contains the syntax of the parameter to the web method, it can be imported to the ClaySys AppForms application or the administrator can define the xml using the following screen. The screen will be opened when the administrator will click on the button named XML next to the parameter textbox.

      Define
      Define XML Syntax for the parameter

      Once the administrator user clicks on the button named “Load XML” it will allow the administrator to open an xml file, the second option is to define the XML manually.

       

      Define
      Define XML Syntax for the parameter

       

      After defining the XML syntax for the parameter, the administrator will be able to map the controls in the form, global variables or session variables to each node value or attribute of a node. This option will be available when the tab XML Tree is clicked.

      Map
      Map control value to Node or attribute

      Configuring the return type to a form

      The return type of a web service can be a single value or a XML.

      1. In case of a single value, the admin user can assign the return value to a Control in the form.
      2. In case of an xml, he will be able to pass the node values to more than one Control in the form.
      3. If the return type is XML, then the admin user will have to execute the web method once with a valid parameter, so that the application can get a sample XML which can be used for the configuration.
      4. Once the application gets a sample xml, the admin can map each node of the xml to any of the controls in the form. This can also be mapped to the datasource of a combo box or lookup etc.

       

      Test
      Test Invocation of the Web method to get a sample output which can be used for the configuration

       

      Output
      Output of the invocation (XML result)
      Column
      Column Mapping of the XML Nodes

      Dynamic Invocation of Web Methods
      The ClaySys AppForms rendering engine will be able to invoke the web method dynamically. There will be a proxy WFC which will be used by the ClaySys AppForms application. The ClaySys AppForms application will only make call to this WCF service which in turn will make the dynamic call to the third party WSDL files from the server.

      Each web service call details will be saved to a transaction table irrespective of the output. The details saved will be timestamp of the call, parameters, return value, etc.

      Failures and Timeouts
      Each WSDL call will be having a timeout period, which will be set to 5000 milliseconds (5 Seconds) by default. When the WSDL timeouts, the datasource will either send the error code to the client or read the values from the database (which can be configured when the datasource is details are saved). The status of the WSDL execution will be passed to the client.

      Exception Handling
      The WSDL configuration will allow user-friendly exception messages to be passed from server to the client. The exception, with the details like Timestamp, User Name, etc. will also be logged to an Exception table so that it can be checked in later stage.

      This module will be using the generic exception handling method that is supported in ClaySys AppForms application for logging the errors to the exception table and sending the user a user friendly error message.

      WSDL Execution Approaches

        1. Real-time execution
      1. Queuing

      Real-time execution
      In this approach the WSDL will be invoked at real time. There will be a proxy ClaySys AppForms service which will be used to invoke any WSDL from the server at real-time. The client will not make the WSDL call directly and will always be using the Proxy service.

      Queuing
      In this model, the ClaySys AppForms application will be using the default datasource to save the data to a SQL table. There will be a server side windows workflow service, which will be doing a data pooling to this table. Each time it finds a new record, the service will read the record and post it to the WSDL. The timeout, exception handling will be logged to the table which will be similar to the real-time execution.

      Attaching Procedure Call
      For each WSDL configured, there can be a procedure call at each successful execution. The procedure call can be another WSDL or a stored procedure which will consume the output of the existing WSDL to do some other activities like merging the records.

  • Maintaining high availability and setup disaster recovery in ClaySys AppForms

    ClaySys AppForms High Availability Configuration

    c3
    This blog will present the High Availability options available to deploy the ClaySys AppForms Solution onto an infrastructure, with the right operational procedures that can provide 24/7 ClaySys AppForms System Availability.

    The High Availability Architecture described in this section, once combined with the right Operational Procedures and Resources, will ensure that Client Maintains a very high level of service uptime.

    Note that, the Microsoft High Availability Technology Options enables seamless failover and load balancing, without needing manual intervention for executing the failover or transferring Network Traffic Load. But proper Operational Procedures need to be in place, to ensure appropriate testing, software maintenance etc.

    To understand the ClaySys AppForms System High Availability Configuration, first we going to review on architecture for High Availability with the SQL Server Database and then we will review on High availability configuration for the ClaySys AppForms Application Servers.

    ClaySys AppForms Database – High Availability

    SQL Server 2008 64 bit (Std or Ent) will be installed and configured for the ClaySys AppForms Solution.

    With SQL Server, Microsoft has two options available for maintaining a high availability configuration. They are SQL Server Failover Clustering and Database Mirroring.

    Failover clustering is a tried and tested technology that uses a shared storage (usually a SAN) between two or more SQL Server nodes that are on different physical servers.
    There are two weaknesses with this approach, which are that the SAN can become a single point of failure, and that failing over from one node to another (E.g. For scheduled maintenance), can take one to two minutes on a busy cluster. This can make it much more difficult to achieve five nines availability, since just applying monthly Microsoft security patches onto each node, and then failing over will probably eat up the six minutes per year of down time with 99.999 availability.

    The alternative that is now becoming more and more popular is for organizations to use Database Mirroring instead of clustering. With database mirroring, you are mirroring at the database level and not the instance level. You will have a principal database and a mirror database, each running on a different server, with its own dedicated storage. This scenario eliminates the single point of failure of the shared storage, and also gives you the ability to failover in 10-15 seconds.

    The only downside is that you will need twice the storage space. It also requires a third server to be the Witness server if you want to run in Synchronous mirroring mode with automatic failover. The witness server can run SQL Server Express Edition and can be a low-end server, which can help on the cost front. If the Witness Server goes down, then the only impact is, we have to manually manage the Failover from the Principal to the Mirror Server.

    The presentation below presents the summary of how SQL Server Mirroring works and the schematic below gives a brief understanding as well:


    SQL Server Mirroring 

    c3
    To understand how exactly mirroring works, please also go through the detailed documentation on the Microsoft website, using the link below:
    https://msdn.microsoft.com/en-us/library/bb934127.aspx


    ClaySys AppForms Application Server – High Availability

    The ClaySys AppForms Application Server will be installed on Windows Server 2008 64 bit edition. We will be setting up a Network Load Balanced cluster of multiple ClaySys AppForms Application servers, to meet Scalability and High Availability requirements for the Client.

    Network Load Balancing (NLB) is a way to configure a pool of machines so they take turns responding to requests.  It’s most commonly implemented in server farms with identically configured machines that spread out the load for a web application.  It can also be used for a firewall (ISA) farm, VPN access points etc.

    Anytime you have TCP/IP traffic, that has become too much load for a single machine, but you still want it to appear as a single machine for access purposes, is when NLB becomes an appropriate fit.

    NLB works in the following manner:

    After you install NLB on a server, you add two or more machines to a NLB Cluster. The machines are configured with 2 IP addresses:  their own private unique one, and a second one that is shared by all the machines in the cluster.

    The machines all run an algorithm that determines whose turn is next at responding to requests.  They also exchange heartbeats with one another, so they all know if one server goes down and won’t allocate any more requests to it.

    You can configure how requests are allocated. You can set up affinity so that requests from one subnet will be responded to from Server A whenever possible, and that other subnets prefer Server B.  Also you could setup 80% of all incoming requests to be handled by Server A, with the rest going to server B.

    There are a few other things you can do with affinity:

    Single affinity: connections initiated by a given IP address are handled by the same server in the cluster until cluster membership changes.

    No affinity: connections are load-balanced based on originating address and port. This is more efficient, as connections from the same client can be routed to several hosts.

    VPN and IPSec affinity: vpn and ipsec sessions will be preserved even if cluster membership changes.

    Class C affinity: useful when internet clients access the cluster through proxies that expose the same class-C addresses. Load balancing is based on the class-C subnet portion of the incoming address.

    The presentation below presents the summary of how NLB works and the schematic below gives a brief understanding as well:


    Network Load Balancing Schematic

    c3


    To understand how exactly NLB works, please also go through the detailed documentation on the Microsoft website, using the link below:
    https://technet.microsoft.com/en-us/library/cc770625(WS.10).aspx

    *Note: The combination of NLB and SQL Server Mirroring will ensure 24/7 High Availability for ClaySys AppForms Services. These technology options will have to be supported by Operational Processes that ensure the right resources are available for maintenance of the ClaySys AppForms Services, and for supporting the Microsoft High Availability options presented in this architecture. 

    ClaySys AppForms Disaster Recovery Setup


    c3

    This section highlights how ClaySys AppForms Disaster Recovery can be setup at the Client, to ensure that a warm standby server infrastructure is being maintained at a separate Disaster Recovery Center.

    So in the event of the ClaySys AppForms Services becoming unavailable form the primary ClaySys AppForms Hosting Service. There could be immediate transition of ClaySys AppForms Services to the Disaster Recovery Infrastructure.

    ClaySys AppForms Disaster Recovery through Log Shipping

    In this scenario, a Disaster Recovery (DR) Server Infrastructure will be maintained by the client at a separate DR site. The application servers will have the same version of the ClaySys AppForms Software installed for both the OS as well as ClaySys AppForms Application.

    The DR is maintained in synch with the production ClaySys AppForms, by synchronizing the Database at the DR site with the database of the Production site in near real time.

    Log shipping is the process of automating the backup of database and transaction log files from a production SQL Server, and then restoring them onto a standby SQL Server at a DR site. The key feature of log shipping is that it will automatically backup transaction logs throughout the day (for whatever interval you specify) and automatically restore them on the standby DR SQL Server. This in effect keeps the two SQL Servers in synch. So should the production server fail, all you have to do is point the users to the new server, to resume services.

    Note that there will always be the tail end of the SQL Database Log files from the Production server that will need to be restored onto the DR Server, if you want to prevent any data loss prior to resumption of services.

    In case there is a catastrophic failure at the Production SQL Server environment, and there is no availability of the tail end of the Database logs, then the maximum risk of Data Loss is as of the last shipment of the transaction log files to the DR Server. Log shipping frequency can be set to minutes, to ensure a minimal data loss in such a scenario.

    The schematic below shows how the SQL Transaction Logs from the Primary SQL Server (Production Environment) are backed up onto a Secondary SQL Server at the client DR Site:

    c3 To understand how exactly SQL Server log shipping works, please also go through the detailed documentation on the Microsoft website, using the link below:

    http://msdn.microsoft.com/en-us/library/ms187103.aspx