Version 20 of WebSocket

Updated 2011-10-13 22:44:19 by jbr

WebSockets are a nice alternative to XMLHTTPRequest for bi-directional communication between a web browser and a server application. They need a browser, e.g. Chrome, that supports the WebSocket API and a server that supports the WebSocket protocol.

jbr 2011-05-1 If someone is adding attributions to the wiki I wish that the would get it right. If you are not absolutely sure who added a comment then please leave it be.

agb Dec. 2010. Chrome now supports an updated version of the websocket protocol so the wibble example previously here no longer works (to see it have a look in this page's history) . The changes to get the current version working are non-trivial.

jbr 2010-12-20 - Here is code that will allow wibble to handshake with the new spec. The version of Chrome that I have (8.0.552.231) has the new handshake but sends the old data framing. I can send data from the client, but, I haven't gotten it to accept data messages from the server. Wibble.tcl needs to be patched to add a way for it to release the socket that will be the websocket channel without responding and closing it:

agb 2010-12-21 - I made a small change to ::wibble::ws-handle to check that chan read actually reads a byte. With this change I have successful, bi-directional messages over the web socket with chrome 8.0.552.224. Thanks for updating wibble.

 # Abort processing on this client.
 proc wibble::abortclient {} {
    return -code 7
 }

AMG: I take it that this command is to be called by a zone handler in order to get Wibble to terminate the coroutine without closing the socket. Correct?

Also, see my comments on Wibble wish list ([L1 ]) for an alternative, less invasive approach.

jbr: Set keepalive 0 at the top of ::wibble::process and then change the exceptional return handling like this:

     } on 7 outcome {
        set keepalive 1
     } finally {
        if { !$keepalive } {
            catch {chan close $socket}
        }
     }

AMG: Some time after you wrote the above, I have changed Wibble to have customizable cleanup handlers. With the latest version of Wibble, instead of modifying the finally block, change the initialization of the cleanup list (top of [process]) to the following:

    set cleanup {
        {chan close $file}
        {if {!$keepalive} {chan close $socket}}
        {dict unset ::wibble::icc::feeds $coro}
    }

jbr: Add this to the zone handlers:

  wibble::handle /ws websocket handler ws-demo

This is your server side callback:

 proc ::ws-demo { event sock { data {} } } {
    switch $event {
        connect {}
        message {
            puts "WS-Demo: $event $sock $data"
        }
    }
  
  ::wibble::ws-send $sock "Hello"
 }

AMG: Are connect and message the only two events that can happen?

jbr: Connect and message are the only two events. WebSockets is a very low level thing (data packets) with the application specific messaging completely undefined.

jbr: Utility to help the server send data frames, doesn't work yet!!

 proc ::wibble::ws-send { sock message } {
    # New data framing?
    #puts -nonewline $sock [binary format cc 4 [string length $message]]$message

    # Old data framing?
    puts  -nonewline $sock "\x00"
    puts  -nonewline $sock $message
    puts  -nonewline $sock "\xFF"

    flush $sock
 }

Handler to accept data from browser. Uses old data framing.

 proc ::wibble::ws-handle { handler sock } {
    if { [chan eof $sock] } {
        puts "Closed $sock"
        close $sock
    } else {
        set code [read $sock 1]
        if {[binary scan $code c code]} {        ; # Do I need this? I think so.
          switch $code {
            0 {
                set message {}

                while { [set c [read $sock 1]] != "\xFF" } {
                    append message $c
                }
                $handler message $sock $message
            }
            default {
                puts "Bad Blocking: $c"
            }
          }
       }
    }
 }

The Zone Handler

 package require md5
  
 proc ::wibble::websocket { state } {
    set upgrade    {}
    set connection {}
    dict with state request header {}

    if { $connection ne "Upgrade" || $upgrade ne "WebSocket" } {
        return
    }

    set sock [dict get $state request socket]

    puts "WebSocket Connect: $sock"

    set key1 [regsub -all {[^0-9]} ${sec-websocket-key1} {}]
    set spc1 [string length [regsub -all {[^ ]}   ${sec-websocket-key1} {}]]
    set key2 [regsub -all {[^0-9]}   ${sec-websocket-key2} {}]
    set spc2 [string length [regsub -all {[^ ]} ${sec-websocket-key2} {}]]

    set key3 [read $sock 8]

    set handler [dict get $state options handler]
    chan event $sock readable [list ::wibble::ws-handle $handler $sock]

    set key1 [expr $key1/$spc1]
    set key2 [expr $key2/$spc2]

    set challenge [binary format II $key1 $key2]$key3
    set response  [md5 $challenge]

    puts $sock "HTTP/1.1 101 WebSocket Protocol Handshake"
    puts $sock "Connection: Upgrade"
    puts $sock "Upgrade: WebSocket"
    puts $sock "Sec-WebSocket-Origin: http://localhost:8080"                ; # This shouldn't be hard coded!!
    puts $sock "Sec-WebSocket-Location: ws://localhost:8080/ws/demo"
    puts $sock ""

    chan configure $sock -translation binary
    puts $sock $response
    chan flush $sock

    $handler connect $sock  ; # There should be an option to pass a session Id here.

    abortclient
 }

AMG: Thanks for the code, guys. I will need to ponder some more before integrating this into Wibble, but I do think I want this feature. However, I think it would benefit from tighter integration. As far as I can tell, it leverages Wibble for establishing the connection but then takes over all I/O. This concept is quite similar to something JCW shared with me the other day, namely an implementation of Server-Sent Events [L2 ] [L3 ]. Whatever I do, I would like it to support both protocols, or at least their common requirements.

If you're wondering why I haven't integrated all this sooner, it's because AJAX was my priority. It may be terribly clumsy compared to WebSockets and Server-Sent Events, but it also has the most browser support.

jcw Neat... jbr's return 7 and keepalive idea look like a very useful tweak:

jbr 2011-05-01 Andy has offered a better way to handle this by removing the socket from the coroutines list and returning an uncaught error. No need to hack Wibble's main zone handler body.

     } on 7 outcome {
        set keepalive 1
     } finally {
        if { !$keepalive } {
            catch {chan close $socket}
        }
     }

Better than what I'm doing right now, which is to do an "icc get" to grab control over the socket by suspending the co-routine indefinitely. The problem with that is that I always get an error on socket close, as wibble tries to resume and send a response to the (now closed) socket. What's not clear to me is whether the "return 7" also causes the request's co-routine to be cleaned up right away (seems like a good idea).

AMG: The coroutine will always be cleaned up, thanks to the "finally" clause inside [process]. The only way to avoid the "finally" clause is to delete the current coroutine command (rename [info coroutine] "") then yield.

A few days ago I came up with another approach that I prefer to any presented on this page or the Wibble wish list: define a new key in the response dict that defines a custom I/O handler that [process] will execute instead of doing its normal post-[getresponse] activities. This way, more of the Wibble infrastructure is available to the custom code: error handling, automatic cleanup, and the ability to loop again and get another HTTP request from the same socket.


2011.1013 jbr Here we are almost a year later with an update.

WebSocket has again moved to a new handshake & framing. Here is a zone handler for Andy's newest wibble and chrome 14 websockets.

 package require sha1
 package require base64

 # Utility proc to frame and send short strings up to 126 chars
 #
 proc ::wibble::ws-send { sock message } {
    puts -nonewline $sock [binary format cc 0x81 [string length $message]]$message
    flush $sock
 } 

 # WebSocket handler proc to receive short (up to 126 chars) text format frames
 #
 proc ::wibble::ws-handle { handler sock } {
    puts ws-handle

    if { [chan eof $sock] } {
        puts "Closed $sock"
        close $sock
    } else {
        binary scan [read $sock 1] c opcode
        binary scan [read $sock 1] c length

        set opcode [expr $opcode & 0x0F]
        set length [expr $length & 0x7F]

        binary scan [read $sock 4]       c* mask
        binary scan [read $sock $length] c* data

        set msg {}
        set i    0
        foreach char $data {
            append msg [binary format c [expr { $char^[lindex $mask [expr { $i%4 }]] }]]
            incr i
        }       
            
        $handler message $sock $msg
    }
 }

 # Zone handler
 #
 proc ::wibble::websocket { state } {
    set upgrade    {}
    set connection {}

    dict with state request header {}

    if { $connection ne "Upgrade" || $upgrade ne "websocket" } {
        return
    }

    set sock [dict get $state request socket]

    puts "WebSocket Connect: $sock"

    set response [base64::encode [sha1::sha1 -bin ${sec-websocket-key}258EAFA5-E914-47DA-95CA-C5AB0DC85B11]]
    set handler  [dict get $state options handler]

    puts $sock "HTTP/1.1 101 WebSocket Protocol Handshake"
    puts $sock "Upgrade:    websocket"
    puts $sock "Connection: Upgrade"
    puts $sock "Sec-WebSocket-Accept: $response"
    puts $sock ""

    chan configure $sock -translation binary
    chan event     $sock readable [list ::wibble::ws-handle $handler $sock]

    $handler connect $sock

    return -code 7
 }

At the top of the ::wibble::process proc I initialize the keepsock variable and modified the cleanup procs like this:

    set keepsock 0
    set cleanup {
        {chan close $file}
        { if { !$keepsock } { chan close $socket } }
        {dict unset ::wibble::icc::feeds $coro}
    }

Then I added a clause in the try structure to catch the return -code 7 from the zone handler:

    } on 7 outcome {
            set keepsock 1
    } finally {

Works for me. Thanks Andy.