Go

Programming language developed by Google: Go

• Similar to C
• Static type system
• Higher-order functions
• Garbage collection
• Object orientation through type interface (no classes but methods can be attached to types)
• Support for concurrency and communication
  • Lightweight threads
  • Communication through channels
    • Formal foundations: Communicating Sequential Processes, Sir Charles Antony Richard Hoare
  • Philosophy: “Do not communicate by sharing memory. Instead, share by communicating.”

Hello World

package main

import "fmt"

var x int

func hi(y int) {
    fmt.Printf("hi %d\n",y)
}

func main() {
    x = 1
    hi(x)
    fmt.Printf("hello, world\n")
}

• Type declarations in ’logical” order
  • var varName varType
  • A variable varName of type varType
• One statement per line. Semicolons are redundant.

Go Toolchain

• Execution with the “command line”

  • go run hello.go

  • gofmt hello.go

    • Automatic “pretty printer”
    • Outputs to the concole by default
    • gofmt -w hello.go writes to the same file

  • You are free to choose the editor (vim, …)

• Or with a web browser on the official Go website

• Or simply find a Go plugin for an IDE.

For your information: our programs always belong to a single file.

Concurrency (goroutine)

Concurrent execution: “just say go”

package main

import "fmt"
import "time"

func thread(s string) {
    for {
        fmt.Print(s)
        time.Sleep(1 * 1e9)
    }
}

func main() {

    go thread("A")
    go thread("B")
    thread("C")
}

• go expression starts a new thread to run expression

• expression must be a function call or a method call; it cannot be parenthesized

• The new thread executes concurrently to the following statements, so in the end thread("A") and thread("B") run concurrently to thread("C")

• The final thread("C") runs in the main thread.

• Threads may run interleaved or on different CPUs (managed by the run-time system)

• As soon as the main thread terminates, all threads started by the main thread are terminated

• Go calls threads goroutines, other languages use fork or spawn instead of go

Concurrency versus Parallelism

• Different goals

• Parallelism: Make programs run faster by making use of additional CPUs (parallel hardware)

• Concurrency: Program organized into multiple threads of control. Threads may work independently or work on a common task.

• See also here and here

Multi-threading in Go

Terminology

Thread = independently sequentially executing code

Thread state:

• Running (currently executing)

• Waiting (ready to execute, but no CPU is available)

• Blocked (waiting for thread-external condition)

Multithreading = Alternating execution of multiple threads on one CPU

Scheduling = Strategy to switch between running and waiting threads

Preemptive scheduling = Every thread gets a certain slice of time to run, then it is preempted and a waiting thread is selected to run

Cooperative scheduling = A thread runs until a blocking command is encountered, then a waiting thread is selected to run

Blocking commands:

• Making the thread sleep (delay/sleep)

• Receiving from a channel (potentially blocking as the channel may be ‘empty’)

• Sending on a channel (potentially blocking as a channel may be ‘full’)

State-based execution

Notation similar to the execution of UPPAAL/communicating automata.

• The program state consists of the states of individual threads. For example:

(Main.Running, A.Waiting, B.Waiting)

describes the state in which

1. the main thread runs, and

2. threads A and B are waiting.

• Creation of a thread (via the go keyword) adds a new thread, initially in wait-state.

Consider the following example:

func a() {
  }
func main() {
 go a()
}

Initially the program state is as follows:

Main.Running

After executing go a(), the state is as follows:

(Main.Running, A.Waiting)

• A path of execution is described by a sequence of individual program states. The transition between the current and the next states is indicated with -->.

      Main.Running

-->   (Main.Running, A.Waiting)

Example

Execution of the program “just say go”. Assumption: one CPU is available.

    Main.Running

--> (Main.Running, A.Waiting)

--> (Main.Running, A.Waiting, B.Waiting)

--> (Main.Blocked, A.Waiting, B.Waiting)

• Main thread is blocking due to a sleep command

• One of the waiting threads gains control

• Assumption (scheduling strategy): the longest-waiting thread gains control

...

--> (Main.Blocked, A.Waiting, B.Waiting)

--> (Main.Blocked, A.Running, B.Waiting)

--> (Main.Waiting, A.Blocked, B. Waiting)

• Thread A blocks due to a sleep command

• In the meantime, the blocking of the main thread has been lifted, because the sleep time is over

...

--> (Main.Waiting, A.Blocked, B. Waiting)

--> (Main.Waiting, A.Blocked, B.Running)

etc.

Lambda’s (anonymous functions) in Go.

Revisiting the “just say go” example.

// Example with "lambda's" = anonymous functions in Go

package main

import "fmt"
import "time"

func thread(s string) {
    for {
        fmt.Print(s)
        time.Sleep(1 * 1e9)
    }
}

func main() {

    // Immediate execution of an anonymous function
    go func() {
        for {
            fmt.Print("A")
            time.Sleep(1 * 1e9)
        }

    }()

    // bFunc is a variable of function type!
    // Type automatically inferred
    bFunc := func() {
        for {
            fmt.Print("B")
            time.Sleep(1 * 1e9)
        }

    }
    go bFunc()
    thread("C")
}

Communication (“channels”)

Threads can communicate using channels, a new datatype in ‘go’. A value sent or received over a channel is called message. A channel can be unbuffered or buffered. A buffered channel can hold a finite number of messages in its buffer.

The following principles hold:

1. A thread can send and receive messages on any channel it holds.

2. A message can be received by exactly one thread.

3. A recipient must necessarily wait for a message, unless a buffered message is available.

4. A sender can continue, as long as the channel still has a buffer available. If the buffer is full (or the channel is unbuffered), the sender is blocked until a message is received from the channel.

See below for details.

Typed Channels

var ch chan int

We declare a variable ch as a channel. The values sent over this channel must be typed int.

Channel creation

ch = make(chan int)

We create a new channel using make. (Analogous to creating a new object.) The declaration var ch chan int attaches a closed channel to ch on which no operations can be executed.

Channel without/with buffer

There are two kinds of channels in Go: without buffer and with buffer. A buffer is a queue of messages.

ch1 = make(chan int)

ch2 = make(chan int, 50)

Channel ch1 is an unbuffered channel. Channel ch2 has buffer space for at most 50 messages.

The following rules hold for exchanging messages:

• Channel without buffer (synchronous communication):

  • A recipient blocks until there is a sender.
  • Similarly for the sender (as there is no buffer available)

• Channel with buffer (asynchronous communication):

  • A recipient blocks if there is no message available in the buffer.
  • A sender blocks only if the buffer is full.
  • The buffer is organized as a queue (FIFO).

Hence, the difference is as follows:

• For an unbuffered channel, a sender always has to synchronize with a recipient. Sender and recipient always block. The Go runtime system checks if there are blocking sender and recipient for the same channel. If so, they communicate with each other and become unblocked.

• For a buffered channel, the sender behaves asynchronously and tries to write the message to the buffer. The sender only blocks if the buffer is full, then it tries again. The recipient always synchronizes with the buffer. If the buffer is empty, the recipient blocks. Otherwise, a message is taken from the buffer.

Sending

ch <- y

Send value y on channel ch (a statement)

Receiving

x = <- ch

Receive from channel ch and save the value in x (an expression, i.e., <- is a unary operator on channels)

Example

package main

import "fmt"
import "time"

func snd(s string, ch chan int) {
    var x int = 0
    for {
        x++
        ch <- x
        fmt.Printf("%s sends %d \n", s, x)
        time.Sleep(1 * 1e9)
    }

}

func rcv(ch chan int) {
    var x int
    for {
        x = <-ch
        fmt.Printf("received %d \n", x)

    }

}

func main() {
    var ch chan int = make(chan int)
    go snd("A", ch)
    rcv(ch)

}

Execution of the example (state-based)

    rcv.Running

--> (rcv.Running, snd.Waiting)

--> (rcv.Blocked_(<-ch?), snd.Waiting)

    Notation: in case of Blocked, the 'subscript' gives the reason

    <-ch?   Recipient is blocked
    ch<-1?  Sender is blocked

--> (rcv.Blocked_(<-ch?), snd.Running)

--> (rcv.Blocked_(<-ch?), snd.Blocked_(ch<-1?))

    First thread waits to receive a message.
    Second thread tries to send a message.

    We say that both threads can synchronize (a kind of "handshake" takes place).
    The message-exchange takes place and the threads are unblocked.

--> (rcv.Waiting, snd.Waiting)

--> (rcv.Running, snd.Waiting)

...

We consider the following variant (1 recipient, 2 senders).

func main() {
    var ch chan int = make(chan int)
    go snd("A", ch) // snd1
    go snd("B", ch) // snd2
    rcv(ch)

}

    rcv.Running

--> (rcv.Running, snd1.Waiting)

--> (rcv.Running, snd1.Waiting, snd2.Waiting)

--> (rcv.Blocked_(<-ch?), snd1.Waiting, snd2.Waiting)

--> (rcv.Blocked_(<-ch?), snd1.Running, snd2.Waiting)

--> (rcv.Blocked_(<-ch?), snd1.Blocked_(ch<-1?), snd2.Waiting)

    Multiple possibilities:
    
    (1) rcv synchronizes with snd1, or
    (2) snd2 thread continues.

    We choose possibility (2)

--> (rcv.Blocked_(<-ch?), snd1.Blocked_(ch<-1?), snd2.Running)

--> (rcv.Blocked_(<-ch?), snd1.Blocked_(ch<-1?), snd2.Blocked_(ch<-1?))

    Multiple possibilities:
    
    (1) rcv synchronizes with snd1, or
    (2) rcv synchronizes with snd2.

    We choose possibility (1)

    [Background: The Go runtime system manages blocking recipients in a queue, so possibility (1) is most likely.]

--> (rcv.Waiting, snd1.Waiting, snd2.Blocked_(ch<-1?))

...

We consider another variant (channel with buffer)

func main() {
    var ch chan int = make(chan int, 1) // channel with buffer
    go snd("A", ch)
    rcv(ch)

}

    rcv.Running

--> (rcv.Running, snd.Waiting)

--> (rcv.Blocked_(<-ch?), snd.Waiting)

--> (rcv.Blocked_(<-ch?), snd.Running)

    // Buffer filled with 1

--> (rcv.Blocked_(<-ch?), snd.Blocked_(Sleep(1s)?))

--> (rcv.Waiting, snd.Blocked_(Sleep(1s)?))

    // Buffer empty again

--> (rcv.Running, snd.Blocked_(Sleep(1s)?))

...

Another variant. Channel with buffer and snd without sleep.

func snd(s string, ch chan int) {
    var x int = 0
    for {
        x++
        ch <- x
        fmt.Printf("%s sends %d \n", s, x)
    }

}

    rcv.Running

--> (rcv.Running, snd.Waiting)

--> (rcv.Blocked_(<-ch?), snd.Waiting)

--> (rcv.Blocked_(<-ch?), snd.Running)

    // Buffer filled with 1

--> (rcv.Blocked_(<-ch?), snd.Blocked_(ch<-2?))

    // Two possibilities
    // (a) rcv reads from channel, or
    // (b) directly from snd
    //
    // Go runtime system chooses variant (a)
    // That is, in case of a buffered channel, the recipient synchronizes with the channel.

--> (rcv.Running, snd.Blocked_(ch<-2?))

    // Channel empty again

--> (rcv.Blocked_(<-ch?), snd.Blocked_(ch<-2?))

    // Again two possibilities
    // (a) snd writes to the channel, or
    // (b) passes the value directly to rcv
    //
    // (a) is the Go variant (see above).

• Using ‘sleep’, execution often gets chaotic (no guarantee that the thread continues after exactly one second)

• Using a channel with buffer, sending is non-blocking (as long as there is space in the buffer).

• Using a channel without buffer, execution is often more predictable, as a sender can only synchronize with a recipient.

Restricted communication

Channel types can be annotated:

Only sending:

func snd(ch chan <- int) {
 ...
}

Only receiving:

func rcv(ch <- chan int) {
 ...
}

Example channel with and without buffer

package main

// Channel without buffer.
// Always gets stuck.
func test1() {
    ch := make(chan int)

    ch <- 1
    <-ch
}

// Channel without buffer.
// Sender synchronizes with recipient.
func test2() {
    ch := make(chan int)

    go func() {
        ch <- 1
    }()
    <-ch

}

// Channel without buffer.
// 2 senders, 1 recipient.
// May get stuck.
func test3() {
    ch := make(chan int)

    snd := func() { ch <- 1 }
    rcv := func() { <-ch }

    go snd()
    go rcv()
    snd()

}

// Channel without buffer.
// 2 senders, 2 recipients.
func test4() {
    ch := make(chan int)

    snd := func() { ch <- 1 }
    rcv := func() { <-ch }

    go snd() // S1
    go snd() // S2
    rcv()    // R1 receives from S1 or S2
    rcv()    // R2
    // If R1 receives from S1, R2 receives from S2
    // If R1 receives from S2, R2 receives from S1

}

// Channel with buffer.
// Does not get stuck.
func test5() {
    ch := make(chan int, 1)

    ch <- 1
    <-ch
}

// Channel with buffer.
// Does not get stuck.
func test6() {
    ch := make(chan int, 2)

    ch <- 1
    ch <- 1
    <-ch
    ch <- 1
}

func main() {
    // test1()
    test2()
    test3()
    test4()
    test3()

}

Synchronous versus Asynchronous Communication

To repeat:

• Channel without buffer (synchronous communication, synchronous channel)
  • Sender blocks if no recipient is available
  • Recipient blocks if no sender is available
  • Direct (synchronous) communication between sender and recipient
  • Sender passes message to recipient
• Channel with buffer (asynchronous communication, asynchronous channel)
  • Sender blocks if buffer is full
  • Recipient blocks if buffer is empty
  • Indirect (asynchronous) communication between sender and recipient
  • Sender puts message in buffer, recipient takes message from buffer

Both modes of communication are equivalent. That is, a channel with buffer can be emulated by channels without buffers. Well-known synchronization primitives (e.g., mutex) can be emulated using channels.

Exercise 1: Mutex

Go supports well-known synchronization primitives like mutex (mutual exclusion) and so on; see http://golang.org/pkg/sync/. However, mutexes and other primitives can be emulated using channels. (Most likely, the mutexes in the Go library are implemented more efficiently, but here we can show that Go could get away with providing only channels).

• Idea: the mutex is represented as a channel with a buffer size of one.
• The buffer is initially empty.
• lock sends and unlock receives.

package main

import "fmt"

type Mutex (chan int)
func newMutex() Mutex
func lock(m Mutex)
func unlock(m Mutex)

var x int

func mySharedVar(y int, m Mutex) {
    for {
        lock(m)
        x = y
        time.Sleep(1e6) // 1ms
        fmt.Printf("%d=%d\n",x,y)
        unlock(m)
    }
}

func testMutex() {
    var m Mutex
    m = newMutex()

    go mySharedVar(1, m)
    mySharedVar(2, m)   
}

Next, try the same thing using a channel without buffer.

Exercise 2: Mutable Variable

Implement a mutable variable, with the following signature:

type MVar (chan int)
func newMVar(x int) MVar
func takeMVar(m MVar) int
func putMVar(m MVar, x int)

• An MVar is either full or empty.
• Initially, an MVar is filled with an integer value.
• takeMVar
  • reads the value if any, and
  • blocks otherwise.
• putMVar
  • writes a value if empty, and
  • block otherwise.
• Hints:
  • takeMVar receives.
  • putMVar sends.
  • Initially, an MVar is full, so takeMVar won’t block at first.
  • The implementation is trivial when we use a channel with buffer size 1. Try the exercise again with a synchronous channel without buffer.

Note: using an MVar, we can easily emulate a mutex. How?

Complete MVar Example

func producer(m MVar) {
    var x int = 1
    for {
        time.Sleep(1 * 1e9)
        putMVar(m, x)
        x++
    }
}

func consumer(m MVar) {
    for {
        var x int = takeMVar(m)
        fmt.Printf("Received %d \n", x)
    }
}

func testMVar() {
    var m MVar

    m = newMVar(1)

    go producer(m)

    consumer(m)

}