Tuesday, August 29, 2017

New Websocket API

A patch released in August, 2017 for Allegro CL 10.0 and 10.1 implements a websocket API which allows users to implement websocket server and client applications in Lisp. The Allegro CL websocket API is described in Websocket API in miscellaneous.htm.
In this note, we provide a very simple example using the API. The websocket module in ACL allows one to implement websocket server and client applications in Lisp (many sites on the web offer more complex demos and tutorials in its use). The websocket protocol is specified in RFC6455. In this example the server is implemented in Lisp and a client is implemented in HTML/Javascript and also in Lisp.
This example uses some Common Lisp source files and some html files. These are all fairly short and all are appended to this article, suitable for cutting and pasting. You can also download a zip file which unzips into a directory named websock/ containing all the files and also an abbreviated text file (simplews.txt) with the instructions from this article.
The example starts a server in one Lisp image. Then it communicates with that server with a browser and also with a client started in another Lisp image. When the server receives a message, its action in response to the message is determined by the on-messagefunction specified in the call to publish-websocket (which is called by the echo-server function defined in server.cl).
The on-message in our example is:
:on-message (lambda (contract data ext)
              (declare (ignore ext))
              (cond ((equalp data "add") (setq *mode* :add))
                    ((equalp data "none") (setq *mode* nil)))
              (case *mode*
                (:add (setq data (concatenate 'string data "...added..."))))
              (websocket-send contract data))
The value of *mode* is initially nil. The server does nothing for any message until the message "add" arrives. Then it responds to that and every subseqeuent message with '[message]...added...' unless a message "none" arrives, in which case *mode* is set to nil and responses stop. This image shows an interaction.

Clicking Open opened the connection. The first two messages ('12345' and 'another) were sent and received but elicited no response. The message 'add' enabled response mode, and messages are responded to with '...added...'. Then the mesasage 'none' turns off responses and subseqeuent messages are not responded to. Clicking Close closes the connection.
This is obviously a simple example but it provides a template for more complex cases.
Here are the steps to running the example:
  1. Start a patched Allegro CL 10.0 or 10.1.
  2. Load the websocket API and (optionally) use the relevant packages:
    (require :websocket)
    (use-package :net.aserve)
    (use-package :net.aserve.client)
    
  3. Edit the host in client and browser source files if necessary. The source files assume the server Lisp, the client Lisp and the browser are all running on the same machine, which can thus be referred to as 'localhost'. If the client Lisp is on a different machine, the client Lisp files must be edited to replace 'localhost' with the actual host name. If the browser is running on a different machine, the html files must be edited to replace 'localhost' with the actual host name.
  4. Start the simple server in a Lisp image by loading the file server.cl and calling the function echo-server:
    :ld server.cl
    (echo-server) ;; add arguments ':port XXXX' if
                  ;; you want a port other than 9001
    
  5. Open a web browser on the file client.html. You should see buttons like those at the top of the illustration above.
Open a connection by clicking Open in the browser. Then send messages. As noted above, sending the message 'add' makes the server respond to messages beyond simply returning them. 'none' stops those responses. The picture above shows a typical interaction. (Occasionally Lisp will print a message about sockets being closed and thus not open-for-output but this seems to be a oddness with some browsers as the communication continues to work.)

Starting a client in a Lisp image

Start a second Lisp image for a Lisp client interaction, load client.cl and execute more forms as shown:
cl-user(5): :ld client.cl
; Loading client.cl
cl-user(6): (echo-client)  ;; add arguments ':port XXXX' if
                           ;; you want a port other than 9001
#<websocket-message-client-contract @ #x201f6c802>
cl-user(7): (echo-send "message1")
:text
cl-user(8): 
RECEIVED message1
(echo-send "message2")
:text
cl-user(9): 
RECEIVED message2
(echo-send "add")
:text
cl-user(10): 
RECEIVED add...added...
(echo-send "after browser")
:text
cl-user(11): 
RECEIVED after browser
(echo-close)
CLOSED with code 1000 
1000
In the server, you can switch to the more elaborate server by calling
(multi-server)
This more complex server maintains a separate application state for each client and illustrates how other event handlers are used. So in the Lisp client, you can do:
cl-user(18): (echo-client :url :multi)
#<websocket-message-client-contract @ #x201fdd592>
cl-user(19): (echo-send "from Lisp client")
:text
cl-user(20): 
RECEIVED from Lisp client
(echo-send "after web add")
:text
cl-user(21): 
Load client2.html into a browser and you can have an interaction as shown in the illustration:

Associated source files

A zipped directory containing all source files used in this example along with the images displayed above and a text file with example instructions can be downloaded from ftp://ftp.franz.com/pub/src/tech_corner/websocket.tar.gz. All the source files are also displayed below suitable for cutting and pasting.
The files are:

server.cl

The port keyword argument in the functions echo-server and multi-server defaults to 9001. Specify :port XXXX in the call to echo-server if you wish to use aother port.
(in-package :user)

(eval-when (compile eval load)
  (require :websocket)
  (use-package :net.aserve))

(defvar *mode* nil)

(defun echo-server (&key (port 9001) debug)
  ;; Simple server with global state.
  (start :port port)
  (setq *mode* nil)
  (publish-websocket
   :path "/simple-echo"
   :on-message (lambda (contract data ext)
   (declare (ignore ext))
   (cond ((equalp data "add") (setq *mode* :add))
         ((equalp data "none") (setq *mode* nil)))
   (case *mode*
     (:add (setq data (concatenate 'string data "...added..."))))
   (websocket-send contract data))
   :debug debug)
  *wserver*)


(defvar *modes* nil)

(defun multi-server (&key (port 9001) debug)
  ;; More complex server that keeps per-client state. 
  (start :port port)
  (setq *modes* (make-hash-table))
  (publish-websocket
   :path "/multi-echo"
   :on-open (lambda (contract)
       (setf (gethash contract *modes*) (list :mode nil)))
   :on-message (lambda (contract data ext
   &aux (plist (gethash contract *modes*)))
   (declare (ignore ext))
   (cond ((equalp data "add") (setf (getf plist :mode) :add))
         ((equalp data "none") (setf (getf plist :mode) nil)))
   (case (getf plist :mode)
     (:add (setq data (concatenate 'string data "...added..."))))
   (websocket-send contract data))
   :on-close (lambda (contract code data)
        (declare (ignore code data))
        (remhash contract *modes*))
   :debug debug)
  *wserver*)

client.cl

In the function echo-client, the port keyword argument defaults to 9001 and the host is specified as 'localhost' in the bindings of 'url'. Specify a different port is the call to echo-client if desired and modify the source to specify a different host if the client Lisp will run on a different host from the server Lisp.
(in-package :user)

(eval-when (compile eval load)
  (require :websocket)
  (use-package :net.aserve)
  (use-package :net.aserve.client))

(defvar *ws* nil)
(defun echo-client (&key (url :simple) (port 9001) debug)
  (case url
    (:simple (setq url "ws://localhost:~A/simple-echo"))
    (:multi (setq url "ws://localhost:~A/multi-echo")))
  (setq *ws*
    (open-websocket (format nil url port)
      :on-message (lambda (contract data ext)
      (declare (ignore contract ext))
      (format t "~&RECEIVED ~A~%" data))
      :on-close (lambda (contract code data)
    (declare (ignore contract))
    (format t "~&CLOSED with code ~A ~A~%"
     code data))
      :debug debug)))

(defun echo-send (text) (websocket-send *ws* text))

(defun echo-close () (close-websocket *ws* :wait t))

client.html

This file refers to 'localhost:9001'. If the browser will be running on a different host from the Lisp server, 'localhost' must be changed to the actual host. If the port used is not 9001, that value too must be edited.
<!DOCTYPE html>

<html>
  <head>
    <title>Echo Chamber</title>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width">
  </head>
  <body>
    <div>
      <input type="text" id="messageinput"/>
    </div>
    <div>
      <button type="button" onclick="openSocket();" >Open</button>
      <button type="button" onclick="send();" >Send</button>
      <button type="button" onclick="closeSocket();" >Close</button>
    </div>
    <!-- Server responses get written here -->
    <div id="messages"></div>
    
    <!-- Script to utilise the WebSocket -->
    <script type="text/javascript">
var webSocket;
var messages = document.getElementById("messages");

function openSocket(){
    // Ensures only one connection is open at a time
    if(webSocket !== undefined && webSocket.readyState !== WebSocket.CLOSED){
        writeResponse("WebSocket is already opened.");
        return;
    }
    // Create a new instance of the websocket
    webSocket = new WebSocket("ws://localhost:9001/simple-echo");
    
    /**
     * Binds functions to the listeners for the websocket.
     */
    webSocket.onopen = function(event){
        writeResponse("CONNECTED ");
    };
    
    webSocket.onmessage = function(event){
        writeResponse("RECEIVED: " + event.data);
    };
    
    webSocket.onclose = function(event){
        writeResponse("Connection closed ");
    };
}

/**
 * Sends the value of the text input to the server
 */
function send(){
    var text = document.getElementById("messageinput").value;
    webSocket.send(text);
    writeResponse("SENT " + text);
}

function closeSocket(){
    webSocket.close();
}

function writeResponse(text){
    messages.innerHTML += "<br/>" + text;
}
    </script>
  </body>
</html>

client2.html

This file refers to 'localhost:9001'. If the browser will be running on a different host from the Lisp server, 'localhost' must be changed to the actual host. If the port used is not 9001, that value too must be edited.
<!DOCTYPE html>

<html>
  <head>
    <title>Echo Chamber</title>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width">
  </head>
  <body>
    
    <div>
      <input type="text" id="messageinput"/>
    </div>
    <div>
      <button type="button" onclick="openSocket();" >Open</button>
      <button type="button" onclick="send();" >Send</button>
      <button type="button" onclick="closeSocket();" >Close</button>
    </div>
    <!-- Server responses get written here -->
    <div id="messages"></div>
    
    <!-- Script to utilise the WebSocket -->
    <script type="text/javascript">
var webSocket;
var messages = document.getElementById("messages");
      
function openSocket(){
    // Ensures only one connection is open at a time
    if(webSocket !== undefined && webSocket.readyState !== WebSocket.CLOSED){
 writeResponse("WebSocket is already opened.");
 return;
    }
    // Create a new instance of the websocket
    webSocket = new WebSocket("ws://localhost:9001/multi-echo");
      
    /**
     * Binds functions to the listeners for the websocket.
     */
    webSocket.onopen = function(event){
 writeResponse("CONNECTED ");
    };
    
    webSocket.onmessage = function(event){
 writeResponse("RECEIVED: " + event.data);
    };
    
    webSocket.onclose = function(event){
 writeResponse("Connection closed ");
    };
}

/**
 * Sends the value of the text input to the server
 */
function send(){
    var text = document.getElementById("messageinput").value;
    webSocket.send(text);
    writeResponse("SENT " + text);
}

function closeSocket(){
    webSocket.close();
}

function writeResponse(text){
    messages.innerHTML += "<br/>" + text;
}
    </script>
  </body>
</html>



Link to the Tech Corner - https://franz.com/support/tech_corner/websocket081717.lhtml

Tuesday, August 1, 2017

Franz (+ Lisp Graph) Newsletter - August 2017



In this issue Intel Monte Story Webcast "Temporality in Graphs" Tech Corner Article: New external format support in Allegro CL 10.1 Recent articles Miller School Researchers Help Push the Limits of Programming Languages in Biology - http://med.miami.edu/news/miller-school-researchers-help-push-the-limits-of-programming-languages-in-/ Classes

Intel Solution Brief - Montefiore Creates Data Analytics to Advance Patient Care
Intel Next Logo
Like all healthcare organizations, Montefiore faces complex challenges from government pressures to reduce costs and stringent regulatory guidelines to diverse patient populations and disruptive technologies. A focus and investment in precision medicine has brought the nation's and industry's attention to expanding the breadth of patient data in order to personalize treatment for individuals and historically underrepresented groups. Further, understanding patients requires information on a complex array of factors, some of which may not even be known during a clinical interaction, such as the home and work environment, nutrition, and genetics...
To read about their solution, see here

Free Webcast - A Time Machine for your Graph
Gruff Icon

August 23, 10 AM Pacific

Many use cases for a Graph Database, like AllegroGraph, involve temporal events. In general, "Events" are modeled as objects that have a start time, end time, a type, some actors and a geospatial location. In this webinar we will explore the temporal aspects of events using a set of practical examples from healthcare, government intelligence, the world of finance, and real estate.
With Gruff v7.0os new "Time Slider" feature, these examples really come to life. Using Gruff, a semantic graph browser, we will visually demonstrate how graphs that are comprised of temporal events are constructed over time, allowing time machine like exploration of your data.
To register for this webcast, see here

Tech Corner Article: New external format support in Allegro CL 10.1
lambda
The recently released Allegro CL version 10.1 has a number of enhancements relating to characters and to external formats. These enhancements include:
  • Support for the GB 18030 external format. The GB 18030 external format is a Chinese government standard which defines the required language and character support necessary for software in China. See this Wikipedia article for more information on GB 18030. See See Basic External-Format Types in iacl.htm for a list of external formats. Support is provided for ensuring writing and reading characters with the :gb18030 external format are valid. See the variable *gb-18030-transcoding-error-action* for more information.
  • Better support for UTF-16 characters. 16-bit character Allegro CL (that is, the alisp, mlisp, and allegro executables, but not alisp8, mlisp8, and allegro8) stores characters in UTF-16. New operators facilitate the creation of UTF-16 octets and characters. These are particularly useful when using characters outside the ASCII range. The operators include code-to-utf16-charcompose-octet-array, and compose-string.
  • Better support for BOMs (Byte Order Marks) in Unicode. A new function sniff-for-unicode applied to a stream returns information on BOMs. The value of the variable *unicode-version* is a string containing the version number for the Unicode Character Database used to build the Allegro CL character name table and collation rules. And new unicode external formats handle BOMs. See The unicode and fat External-Format Types; the unicode BOM in iacl.htm.
See here for additional information

Recent Articles about Franz 














Miller School Researchers Help Push the Limits of Programming Languages in Biology
Computer-Code-Tunnel
Bohdan Khomtchouk, a fourth-year human genetics and genomics Ph.D candidate working in the Center for Therapeutic Innovation and the Department of Psychiatry and Behavioral Sciences at the University of Miami Miller School of Medicine, has spearheaded the publication of a high-impact review paper published in Briefings in Bioinformatics that was the subject of this year’s invited keynote speech at the European Lisp Symposium in Brussels, Belgium.
Khomtchouk is joined in authorship by his graduate advisor, Claes Wahlestedt, M.D., Ph.D., professor of psychiatry and behavioral sciences, associate dean for therapeutic innovation and director of the Center for Therapeutic Innovation at the Miller School; Edmund Weitz, Ph.D., professor of mathematics at the University of Applied Sciences in Hamburg, Germany; and Peter D. Karp, Ph.D., director of the Bioinformatics Research Group within the Artificial Intelligence Center at SRI International.
“Bioinformatics and computational biology software is dominated largely by higher-level languages like R and Python and lower-level languages like C and C++,” said Khomtchouk. “In our paper, we systematically review the advantages posed by a unique hybrid of languages, called the Lisp family of languages, that offer both high-level scripting and low-level performance capabilities not commonly seen in other languages.”
In bioinformatics and computational biology, Lisp has successfully been applied to research in systems biology, high-performance computing, database curation, drug discovery, computational chemistry and nanotechnology, network and pathway -omics analysis, single-nucleotide polymorphism analysis and RNA structure prediction...
Read more here

Follow us on Google Plus, Twitter, LinkedIn, and YouTube 

Google+
Twitter


Training Schedule
Gruff

LabBECOME ALLEGRO CERTIFIED - To obtain your Allegro CL Certification enroll in our LIVE Program which offers developers an opportunity to learn and improve their Lisp programming skills from the comfort of their home or office while interacting with the Franz instructor.
Lisp Programming Series Level I: Basic Lisp Essentials - September 6, 13, and 20
Lisp Programming Series Level II: Specialized Components of Lisp - October 4, 11, and 18
For additional information and to register, see here.