Volatile Variables in Golang

   

In the past, I have used lockless code when there is a single reader and single writer and the reader can accept stale information to some extent (for example, this sample code on my github) .

However, there is a problem in this code. If the two go routines run on different cpu cores, then the reader go routines may never get changes from writer go routine? So, the solution I typically employed in languages such as C/C++ is to use volatile type for flag. However, go doesn't seem to support volatile variables as per these discussions: google groups discussion, stackoverflow discussion!! 

So, the solution for the above situation is to use an atomic variable. To find the correct way of writing the above code, please check here.

Skipping Go Race detector for some packages

  

In my previous post, I covered the basics of Go Race detector. In most Golang, projects, we would have multiple projects. 

For some reason, if we want to skip race detector for some packages, we can use this option as the first line of the package & an empty second line:

// +build !race

I ran the sample code on my github with the above option to get this:

Babus-MacBook-Pro:Race_Tests_inGolang babuneelam$ go build -race

package racetest: build constraints exclude all Go files in /Users/babuneelam/Projects/GoProjects/Race_Tests_inGolang

Babus-MacBook-Pro:Race_Tests_inGolang babuneelam$ 

I have only a single package in this, so this error. If I were to have multiple packages in this repository, it would have build those.

Exploring Go Race Detector

   

I got to know about Golang Race Detector from a colleague & wanted to give this a try. Here is the sample code on my github  I tested (also, same code on Go playground for quicker testing).

We can use race detector in multiple ways:

1) Using go run -race with the specific go file:

Babus-MacBook-Pro:Race_Tests_inGolang babuneelam$ go run -race main.go

flag:  true

==================

WARNING: DATA RACE

Write at 0x0000012742a1 by goroutine 7:

  main.writer()

      /Users/babuneelam/Projects/GoProjects/Race_Tests_inGolang/main.go:12 +0x55

Previous read at 0x0000012742a1 by main goroutine:

  main.reader()

      /Users/babuneelam/Projects/GoProjects/Race_Tests_inGolang/main.go:19 +0x3e

  main.main()

      /Users/babuneelam/Projects/GoProjects/Race_Tests_inGolang/main.go:27 +0x5e

Goroutine 7 (running) created at:

  main.main()

      /Users/babuneelam/Projects/GoProjects/Race_Tests_inGolang/main.go:26 +0x59

==================

flag:  true

^Csignal: interrupt

Babus-MacBook-Pro:Race_Tests_inGolang babuneelam$ 

 

2) Build the go program with "-race" flag:

Babus-MacBook-Pro:Race_Tests_inGolang babuneelam$ go build -race

Babus-MacBook-Pro:Race_Tests_inGolang babuneelam$ 

Then, run the program:

Babus-MacBook-Pro:Race_Tests_inGolang babuneelam$ ./racetest 

flag:  true

==================

WARNING: DATA RACE

Write at 0x0000012742a1 by goroutine 7:

  main.writer()

      /Users/babuneelam/Projects/GoProjects/Race_Tests_inGolang/main.go:12 +0x55

Previous read at 0x0000012742a1 by main goroutine:

  main.reader()

      /Users/babuneelam/Projects/GoProjects/Race_Tests_inGolang/main.go:19 +0x3e

  main.main()

      /Users/babuneelam/Projects/GoProjects/Race_Tests_inGolang/main.go:27 +0x5e

Goroutine 7 (running) created at:

  main.main()

      /Users/babuneelam/Projects/GoProjects/Race_Tests_inGolang/main.go:26 +0x59

==================

flag:  true

flag:  true

^C

Babus-MacBook-Pro:Race_Tests_inGolang babuneelam$ 

 

3)  go test -race mypkg - TBD

4)  go install -race mypkg - TBD

As we can see the go race detector displayed us two pieces needed:

1. A race warning  
2. The two code locations that are causing the race

So, it's quite useful. To find the correct way of writing the above code, please check here.

However, this is not a static analysis tool. So, we need to run the program to detect the races. And only those races that happen during execution are detected. If we use race as part of test cases, then race detector is only as effective as your test coverage !!

In terms of performance, Golang Race Detector says this:

The cost of race detection varies by program, but for a typical program, memory usage may increase by 5-10x and execution time by 2-20x.

The race detector currently allocates an extra 8 bytes per defer and recover statement. Those extra allocations are not recovered until the goroutine exits. This means that if you have a long-running goroutine that is periodically issuing defer and recover calls, the program memory usage may grow without bound. These memory allocations will not show up in the output of runtime.ReadMemStats or runtime/pprof.

So, this is definitely not for production environments.

Online IDE for Golang: Go Playground

Golang has an online IDE here: Go Playground. Here we can experiment sample programs and share code snippets with others. As per the discussion in the google groups here, the shared urls seem to be retained forever - at least for now.

The online IDE doesn't autofill methods, imports etc like IDEs like IntelliJ, Eclipse do. That would have been even nicer.

More details about this playground:

• Go Playground Implementation Details
• Source code of go playground

Also, "About" page on the online IDE page has a bit more details such as limitations etc.


TBD:

• How do we run a go program in the online IDE with input arguments?

Setting up a Go project in Vscode

  

Here are the steps I followed to setup a Go project in Vscode:

1) Install Vscode Go plugin.

2) Create a project

3) I still had this error:

4) To fix this, I downloaded the go 1.15 separately.  

5) Set GOPATH - export PATH=$PATH:/usr/local/go/bin/go 

Even after this, I am still not able to run my go program directly from UI. I could only the go program from the terminal.

One of my friends who uses Vscode helped me run go programs from vscode. He suggested these additional steps:

1) Open settings in vscode

 

2) Search for "gopath" & open "settings.json"

 

3) Set goroot & gopath in this file

 

4) After this, I had to restart vscode. After restart, vscode threw some pop ups to install a few things & I proceeded with those:

 

Tools environment: GOPATH=/Users/babuneelam/Projects/GoProjects/go_path

Installing 1 tool at /Users/babuneelam/Projects/GoProjects/go_path/bin in module mode.

gopls  

Installing github.com/mdempsky/gocode SUCCEEDED

Installing github.com/uudashr/gopkgs/v2/cmd/gopkgs SUCCEEDED

Installing github.com/ramya-rao-a/go-outline SUCCEEDED

Installing golang.org/x/tools/gopls SUCCEEDED

All tools successfully installed. You are ready to Go :).

Installing github.com/acroca/go-symbols SUCCEEDED

Installing golang.org/x/tools/cmd/guru SUCCEEDED

Installing golang.org/x/tools/cmd/gorename SUCCEEDED

Installing github.com/cweill/gotests/... SUCCEEDED

Installing github.com/fatih/gomodifytags SUCCEEDED

Installing github.com/josharian/impl SUCCEEDED

Installing github.com/davidrjenni/reftools/cmd/fillstruct SUCCEEDED

Installing github.com/haya14busa/goplay/cmd/goplay SUCCEEDED

Installing github.com/godoctor/godoctor SUCCEEDED

Installing github.com/go-delve/delve/cmd/dlv SUCCEEDED

Installing github.com/stamblerre/gocode SUCCEEDED

Installing github.com/rogpeppe/godef SUCCEEDED

Installing github.com/sqs/goreturns SUCCEEDED

Installing golang.org/x/lint/golint SUCCEEDED

All tools successfully installed. You are ready to Go :).

Tools environment: GOPATH=/Users/babuneelam/Projects/GoProjects/go_path

Installing 1 tool at /Users/babuneelam/Projects/GoProjects/go_path/bin in module mode.

goimports

Installing golang.org/x/tools/cmd/goimports SUCCEEDED

All tools successfully installed. You are ready to Go :).

5) Run & Launch. We can also set breakpoints too now.

 

By setting up vscode, I could avoid paying for Goland/IntelliJ, atleasr for now :)

Hope this helps.

IntelliJ & Go Plugin

  

In the recent times, I have been using IntelliJ Ultimate edition at work and community edition on my personal laptop. 

Tried to create some sample go projects in my personal laptop for my own testing. I thought it would be quite an easy step to create a Go project in IntelliJ. Here are some things I tried:

1) Tried to look for a Go plugin in IntelliJ Plugins, couldn't find it. 

2) Downloaded Go plugin from IntelliJ Go Plugins and followed instructions as provided at "Install Plugins from disk". But, the installation threw these errors:

So, IntelliJ Community edition doesn't allow Go plugins. As per this article from Jetbrains:

it was always in the plan to have Go plugin functionality available as part of the GoLand IDE. If you want Go functionality combined with other technologies, then IntelliJ IDEA Ultimate is the right choice for you

Lack of this information wasted some time for me today. The cost for ultimate edition seems to $249 for individual use. 

JShell Overview

JShell helps developers to quickly test java code snippets in an interactive shell. Many other languages (Python, ..) also have such interactive shells. Java introduced JShell in as part of JDK 9.

Here is a sample JShell session I tried:

I haven't read and tried everything thats possible in JShell, gave a cursory read of https://docs.oracle.com/javase/9/jshell/toc.htm. I felt this resource was quite useful and covers JShell capabilities reasonably.


References:
https://docs.oracle.com/javase/9/jshell/toc.htm
https://developers.redhat.com/blog/2017/10/03/use-jshell-command-line-tool-introduced-jdk-9/

Getting Started with Java in IntelliJ IDE

I have been using VSCode IDE a bit. However, my team at Salesforce uses IntelliJ IDE (Ultimate Edition). So, I decided to give it a try. Here are the steps to get started with a sample project (say, the usual "Hello World !!") in Java.


1) Install Java JDK for Mac: I think it was already installed earlier. So, didn't have to do this step in my case.

2) Install IntelliJ Community Edition: https://www.jetbrains.com/idea/download/#section=mac. I am using MAC. But, if your OS is different, choose the platform appropriately.

3) Install. Move IntelliJ to Applications folder. This created an icon like this in Mac's Launchpad:

                                                           

4) Create a new project in IntelliJ: 

I created and named my Project "HelloWorld". I think the project name should start with an upper case letter?

5) I then created a new class named "HelloWorld" and filled in a few lines of code (some of it getting auto-filled). When I clicked "Run", the program output "Hello World !!" on the console pane.

Hope this helps.

Getting Started with Java in VSCode IDE

I started picking up Java recently. Here are the steps to get started with a sample project (say, the usual "Hello World !!") in Java.


1) Install Java JDK for Mac: I think it was already installed earlier. So, didn't have to do this step in my case.

2) Install VSCode: https://code.visualstudio.com/download. A friend of mine suggested to use VSCode, so I'm going to give it a try.

3) Install. Move VSCode to Applications folder. This created an icon like this in Mac's Launchpad:

                                                              

4) Create a new workspace in VSCode by choosing "Add Workspace Folder". 

I created and named my workspace folder "HelloWorld". 

5) I then created a file named "HelloWorld.java" and filled in a few lines of code (some of it getting auto-filled). After I completed typing public static void main method's prototype, I got "Run | Debug" appear on top of it automatically.

When I clicked "Run", the program output "Hello World !!" on the console pane.

Hope this helps.

How to sync local repository sources from the github repository?

At times, the local sources repository could be obsolete as the gitgub repository changes after we imported. In this post, I will provide the command required to sync the local repository from the github repository.

babu@babu-VirtualBox:~/EKiD$ git pull origin master
remote: Compressing objects: 100% (5/5), done.
remote: Total 5 (delta 3), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (5/5), done.
From https://github.com/babuneelam/EKiD
 * branch            master     -> FETCH_HEAD
   2005d11..46764be  master     -> origin/master
Updating 2005d11..46764be
Fast-forward
 .README.swp | Bin 12288 -> 0 bytes
 README      |  51 ++++++++++++++++++++++++++++++++++++++++++---------
 2 files changed, 42 insertions(+), 9 deletions(-)
 delete mode 100644 .README.swp
babu@babu-VirtualBox:~/EKiD$

TBD: How are conflicts between local and remote changes handled?

Hope this helps.

How to import from and export changes to an existing github repository?

In this post, I will explain how to import sources from an existing github repository to a local directory, make changes locally as necessary and then export the changes back to the github repository.

• Import from your github repository to local directory:

babu@babu-VirtualBox:~$ git clone https://github.com/babuneelam/EKiD.git
Cloning into 'EKiD'...
remote: Counting objects: 54, done.
remote: Compressing objects: 100% (48/48), done.
remote: Total 54 (delta 6), reused 46 (delta 4), pack-reused 0
Unpacking objects: 100% (54/54), done.
Checking connectivity... done.
babu@babu-VirtualBox:~$ cd EKiD/
babu@babu-VirtualBox:~/EKiD$ ls
ConfigControls  LICENSE  README  SampleOutputs  Sources
babu@babu-VirtualBox:~/EKiD$ 

• Make change as necessary

• See the diff

babu@babu-VirtualBox:~/EKiD$ git diff
diff --git a/ConfigControls/EnvCfg b/ConfigControls/EnvCfg
index 485a346..63a1976 100644
--- a/ConfigControls/EnvCfg
+++ b/ConfigControls/EnvCfg
@@ -1,5 +1,5 @@
-GECKO_DRV_PATH=/Users/babuneelam/Downloads/geckodriver
-FONTS_FOLDER=/Library/Fonts
+GECKO_DRV_PATH=/home/babu/bin/geckodriver
+FONTS_FOLDER=/usr/share/fonts/truetype/msttcorefonts/
 #GECKO_DRV_LOG_FILE=RunTimeInfo/geckodriver.log
 #OUTPUT_DIR=RunTimeInfo/output
 #IMAGE_STORE_PATH=RunTimeInfo/google_images
babu@babu-VirtualBox:~/EKiD$ 

babu@babu-VirtualBox:~/EKiD$ git diff --name-only
ConfigControls/EnvCfg
babu@babu-VirtualBox:~/EKiD$ 

• Tell git who you are:

      git config --global user.email "you@example.com"

      git config --global user.name "Your Name"

• TBD: Add/Remove existing files from git repository
• Export ALL the local changes to the github repository:

      babu@babu-VirtualBox:~/EKiD$ git commit -a
        [master 32a66e9] A few fixes
         2 files changed, 3 insertions(+), 3 deletions(-)
      babu@babu-VirtualBox:~/EKiD$

        babu@babu-VirtualBox:~/EKiD$ git pull origin master
        From https://github.com/babuneelam/EKiD
        * branch            master     -> FETCH_HEAD
        Already up-to-date.
      babu@babu-VirtualBox:~/EKiD$ 

      babu@babu-VirtualBox:~/EKiD$ git push -u origin master
        Username for 'https://github.com': babuneelam
        Password for 'https://babuneelam@github.com':
        Counting objects: 6, done.
        Compressing objects: 100% (6/6), done.
        Writing objects: 100% (6/6), 604 bytes | 0 bytes/s, done.
        Total 6 (delta 4), reused 0 (delta 0)
        remote: Resolving deltas: 100% (4/4), completed with 4 local objects.
        To https://github.com/babuneelam/EKiD.git
           46764be..32a66e9  master -> master
        Branch master set up to track remote branch master from origin.
      babu@babu-VirtualBox:~/EKiD$ 

• Export only a few files of the local changes to the github repository:

babu@babu-VirtualBox:~/EKiD$ git commit ConfigControls/AlgoCfg -m 'Documentation fixes'

[master 4668ebb] Documentation fixes

1 file changed, 8 insertions(+)

babu@babu-VirtualBox:~/EKiD$ git commit ConfigControls/FontCfg -m 'Documentation fixes'

[master a3437e1] Documentation fixes

1 file changed, 5 insertions(+), 2 deletions(-)

babu@babu-VirtualBox:~/EKiD$

babu@babu-VirtualBox:~/EKiD$ git pull origin master

From https://github.com/babuneelam/EKiD

* branch            master     -> FETCH_HEAD

Already up-to-date.

babu@babu-VirtualBox:~/EKiD$ git push -u origin master

Username for 'https://github.com': babuneelam

Password for 'https://babuneelam@github.com':

Counting objects: 8, done.

Compressing objects: 100% (8/8), done.

Writing objects: 100% (8/8), 1.02 KiB | 0 bytes/s, done.

Total 8 (delta 3), reused 0 (delta 0)

remote: Resolving deltas: 100% (3/3), completed with 1 local object.

To https://github.com/babuneelam/EKiD.git

  b61c9a5..a3437e1  master -> master

Branch master set up to track remote branch master from origin.

babu@babu-VirtualBox:~/EKiD$

Hope this helps.