InfluxDB with Golang , 1st InfluxDB Meetup at MayaData ,Bangalore

Biradars-MacBook-Air-4:~ sangam$ /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
==> This script will install:
/usr/local/bin/brew
/usr/local/share/doc/homebrew
/usr/local/share/man/man1/brew.1
/usr/local/share/zsh/site-functions/_brew
/usr/local/etc/bash_completion.d/brew
/usr/local/Homebrew

Press RETURN to continue or any other key to abort
==> Downloading and installing Homebrew...
remote: Enumerating objects: 38, done.
remote: Counting objects: 100% (38/38), done.
remote: Compressing objects: 100% (9/9), done.
remote: Total 41 (delta 29), reused 34 (delta 29), pack-reused 3
Unpacking objects: 100% (41/41), done.
From https://github.com/Homebrew/brew
   8072eac0a..a59be43ec  master     -> origin/master
HEAD is now at a59be43ec Merge pull request #6744 from Bo98/bottle-dotfiles
Already up-to-date.
==> Installation successful!

==> Homebrew has enabled anonymous aggregate formulae and cask analytics.
Read the analytics documentation (and how to opt-out) here:
  https://docs.brew.sh/Analytics

==> Homebrew is run entirely by unpaid volunteers. Please consider donating:
  https://github.com/Homebrew/brew#donations
==> Next steps:
- Run `brew help` to get started
- Further documentation: 
    https://docs.brew.sh
Biradars-MacBook-Air-4:~ sangam$ brew update 
Already up-to-date.
Biradars-MacBook-Air-4:~ sangam$ brew install influxdb
Updating Homebrew...
Warning: influxdb 1.7.9 is already installed and up-to-date
To reinstall 1.7.9, run `brew reinstall influxdb`
Biradars-MacBook-Air-4:~ sangam$ ln -sfv /usr/local/opt/influxdb/*.plist ~/Library/LaunchAgents
/Users/sangam/Library/LaunchAgents/homebrew.mxcl.influxdb.plist -> /usr/local/opt/influxdb/homebrew.mxcl.influxdb.plist
Biradars-MacBook-Air-4:~ sangam$ launchctl load ~/Library/LaunchAgents/homebrew.mxcl.influxdb.plist
Biradars-MacBook-Air-4:~ sangam$ influxd -config /usr/local/etc/influxdb.conf

 influxd -config /usr/local/etc/influxdb.conf

 8888888           .d888 888                   8888888b.  888888b.
   888            d88P"  888                   888  "Y88b 888  "88b
   888            888    888                   888    888 888  .88P
   888   88888b.  888888 888 888  888 888  888 888    888 8888888K.
   888   888 "88b 888    888 888  888  Y8bd8P' 888    888 888  "Y88b
   888   888  888 888    888 888  888   X88K   888    888 888    888
   888   888  888 888    888 Y88b 888 .d8""8b. 888  .d88P 888   d88P
 8888888 888  888 888    888  "Y88888 888  888 8888888P"  8888888P"

2019-11-15T17:26:30.221803Z	info	InfluxDB starting	{"log_id": "0J81U2JG000", "version": "v1.7.9", "branch": "master", "commit": "23bc63d43a8dc05f53afa46e3526ebb5578f3d88"}
2019-11-15T17:26:30.221836Z	info	Go runtime	{"log_id": "0J81U2JG000", "version": "go1.13.3", "maxprocs": 4}
2019-11-15T17:26:30.324366Z	info	Using data dir	{"log_id": "0J81U2JG000", "service": "store", "path": "/usr/local/var/influxdb/data"}
2019-11-15T17:26:30.324459Z	info	Compaction settings	{"log_id": "0J81U2JG000", "service": "store", "max_concurrent_compactions": 2, "throughput_bytes_per_second": 50331648, "throughput_bytes_per_second_burst": 50331648}
2019-11-15T17:26:30.324493Z	info	Open store (start)	{"log_id": "0J81U2JG000", "service": "store", "trace_id": "0J81U2i0000", "op_name": "tsdb_open", "op_event": "start"}
2019-11-15T17:26:30.330497Z	info	Reading file	{"log_id": "0J81U2JG000", "engine": "tsm1", "service": "cacheloader", "path": "/usr/local/var/influxdb/wal/_internal/monitor/1/_00001.wal", "size": 741235}
 ......................................................  

open new terminal

Biradars-MacBook-Air-4:~ sangam$ influx
Connected to http://localhost:8086 version v1.7.9
InfluxDB shell version: v1.7.9
> create user opsadmin with password 'pass123'
> grant all privileges to opsadmin
> create database metricsdb
> 

Working on InfluxDB with Go Client

The v2 version of the Go client library for InfluxDB is available from github.com/influxdata/influxdb/ client/v2. The Go client library is maintained by the InfluxDB team. To install the v2 version of the package, run the following command:

go get github.com/influxdata/influxdb/client/v2

Connecting to InfluxDB
By default, InfluxDB listens on port 8086. The following code block connects to InfluxDB with user account opsadmin.

c, err := client.NewHTTPClient(client.HTTPConfig{
                Addr:     "http://localhost:8086",
                Username: “opsadmin”,
                Password: “pass123”,
})

The function NewHTTPClient returns a new InfluxDB Client from the given configuration. Struct type HTTPConfig is used to provide configuration for creating the InfluxDB Client. Here is the definition of struct HTTPConfig:

// HTTPConfig is the config data needed to create an HTTP Client
type HTTPConfig struct {
        // Addr should be of the form "http://host:port"
        // or "http://[ipv6-host%zone]:port".
        Addr string
        // Username is the influxdb username, optional
        Username string
        // Password is the influxdb password, optional
        Password string
        // UserAgent is the http User Agent, defaults to "InfluxDBClient"
        UserAgent string
        // Timeout for influxdb writes, defaults to no timeout
        Timeout time.Duration
        // InsecureSkipVerify gets passed to the http client, if true, it will
        // skip https certificate verification. Defaults to false
        InsecureSkipVerify bool
        // TLSConfig allows the user to set their own TLS config for the HTTP
        // Client. If set, this option overrides InsecureSkipVerify.
        TLSConfig *tls.Config
}

Once you have created an InfluxDB Client, you can use it for both write and query operations.

Writing Points to InfluxDB

When you write points to measurement for persisting data into InfluxDB, you should do so in batches. To write points in batches, you first create a new BatchPoints value as shown here:

bp, err := client.NewBatchPoints(client.BatchPointsConfig{
                Database:  “metricsdb”,
                Precision: "s",
})

A BatchPoints value is created by providing a configuration. The property Precision specifies the precision for timestamp created for each point. By default, all timestamps in Unix are in nanoseconds. If you want to provide timestamps in any unit other than nanoseconds, you must provide the appropriate precision. Use n, u, ms, s, m, and h for nanoseconds, microseconds, milliseconds, seconds, minutes, and hours, respectively. The following code block creates a new point by providing values for tags, fields, and timestamp to the measurement named cpu.

// tagset – “host” and “region”
tags := map[string]string{
 "host":   "host1"
 "region": "us-west"
}
// field - "cpu_usage"
fields := map[string]interface{}{
 "cpu_usage": 46.22
}
// New point to measurement named “cpu”
pt, err := client.NewPoint("cpu ", tags, fields, time.Now())
 if err != nil {
       log.Fatalln("Error: ", err)
}
bp.AddPoint(pt)

Because you write points in batches, you add n number of points to BatchPoints using the function AddPoint. Once all points are added to BatchPoints, call the function Write of the InfluxDB Client instance to complete the write operation.


// Write the batch
c.Write(bp) // c is the instance of InfluxDB Client instance

Reading Points from InfluxDB

InfluxDB provides the ability to query data using familiar SQL constructs. This code block determines the count value of points in measurement cpu.

command:= fmt.Sprintf("SELECT count(%s) FROM %s", "cpu_usage", "cpu")
q := client.Query{
                Command:  command,
                Database: DB,
        }
        // Query the Database
        if response, err := c.Query(q)  // // c is the instance of InfluxDB Client instance
        if err != nil {
              log.Fatalln("Error: ", err)
        }
        count :=response.Results[0].Series[0].Values[0][1]

Writing and Reading of Points to a Measurement “cpu” in InfluxDB

// Meetup - working with InfluxDB
package main

import (
	"encoding/json"
	"fmt"
	"log"
	"math/rand"
	"time"

	_ "github.com/influxdata/influxdb1-client"
	client "github.com/influxdata/influxdb1-client/v2"
)

const (
	// DB provides the database name of the InfluxDB
	DB       = "metricsdb"
	username = "opsadmin"
	password = "pass123"
)

func main() {
	// Create client
	c := influxDBClient()
	// Write operations
	// Create metrics data for measurement "cpu"
	createMetrics(c)
	// Read operations
	// Read with limit of 10
	readWithLimit(c, 10)
	// Read mean value of "cpu_usage" for a region
	meanCPUUsage(c, "us-west")
	// Read count of records for a region
	countRegion(c, "us-west")

}

// influxDBClient returns InfluxDB Client
func influxDBClient() client.Client {
	c, err := client.NewHTTPClient(client.HTTPConfig{
		Addr:     "http://localhost:8086",
		Username: username,
		Password: password,
	})
	if err != nil {
		log.Fatalln("Error: ", err)
	}
	return c
}

// createMetrics write batch points to create the metrics data
func createMetrics(clnt client.Client) {
	batchCount := 100
	rand.Seed(42)

	// Create BatchPoints by giving config for InfluxDB
	bp, _ := client.NewBatchPoints(client.BatchPointsConfig{
		Database:  DB,
		Precision: "s",
	})
	// Batch update to adds Points
	for i := 0; i < batchCount; i++ {
		regions := []string{"us-west", "us-central", "us-north", "us-east"}
		// tagset – “host” and “region”
		tags := map[string]string{
			"host":   fmt.Sprintf("192.168.%d.%d", rand.Intn(100), rand.Intn(100)),
			"region": regions[rand.Intn(len(regions))],
		}

		value := rand.Float64() * 100.0
		// field - "cpu_usage"
		fields := map[string]interface{}{
			"cpu_usage": value,
		}

		pt, err := client.NewPoint("cpu", tags, fields, time.Now())

		if err != nil {
			log.Fatalln("Error: ", err)
		}
		// Add a Point
		bp.AddPoint(pt)

	}
	// Writes the batch update to add points to measurement "cpu"
	err := clnt.Write(bp)
	if err != nil {
		log.Fatalln("Error: ", err)
	}
}

// queryDB query the database
func queryDB(clnt client.Client, command string) (res []client.Result, err error) {
	// Create the query
	q := client.Query{
		Command:  command,
		Database: DB,
	}
	// Query the Database
	if response, err := clnt.Query(q); err == nil {
		if response.Error() != nil {
			return res, response.Error()
		}
		res = response.Results
	} else {
		return res, err
	}
	return res, nil
}

// readWithLimit reads records with a given limit
func readWithLimit(clnt client.Client, limit int) {
	q := fmt.Sprintf("SELECT * FROM %s LIMIT %d", "cpu", limit)
	res, err := queryDB(clnt, q)
	if err != nil {
		log.Fatalln("Error: ", err)
	}

	for i, row := range res[0].Series[0].Values {
		t, err := time.Parse(time.RFC3339, row[0].(string))
		if err != nil {
			log.Fatalln("Error: ", err)
		}
		val, err := row[1].(json.Number).Float64()
		fmt.Printf("[%2d] %s: %f\n", i, t.Format(time.Stamp), val)
	}
}

// meanCPUUsage reads the mean value of cpu_usage
func meanCPUUsage(clnt client.Client, region string) {
	q := fmt.Sprintf("select mean(%s) from %s where region = '%s'", "cpu_usage", "cpu", region)
	res, err := queryDB(clnt, q)
	if err != nil {
		log.Fatalln("Error: ", err)
	}
	value, err := res[0].Series[0].Values[0][1].(json.Number).Float64()
	if err != nil {
		log.Fatalln("Error: ", err)
	}

	fmt.Printf("Mean value of cpu_usage for region '%s':%f\n", region, value)
}

// countRegion reads the count of records for a given region
func countRegion(clnt client.Client, region string) {
	q := fmt.Sprintf("SELECT count(%s) FROM %s where region = '%s'", "cpu_usage", "cpu", region)
	res, err := queryDB(clnt, q)
	if err != nil {
		log.Fatalln("Error: ", err)
	}
	count := res[0].Series[0].Values[0][1]
	fmt.Printf("Found a total of %v records for region '%s'\n", count, region)
}
output :
Biradars-MacBook-Air-4:Influxdb-meetup sangam$ go run main.go
[ 0] Nov 16 06:28:38: 11.901734
[ 1] Nov 16 06:28:38: 15.471216
[ 2] Nov 16 06:28:38: 32.904423
[ 3] Nov 16 06:28:38: 15.973031
[ 4] Nov 16 06:28:38: 88.648864
[ 5] Nov 16 06:28:38: 92.049809
[ 6] Nov 16 06:28:38: 83.304049
[ 7] Nov 16 06:28:38: 18.495674
[ 8] Nov 16 06:28:38: 23.389015
[ 9] Nov 16 06:28:38: 46.009337
Mean value of cpu_usage for region 'us-west':46.268998
Found a total of 27 records for region 'us-west'

influx shell

> influx
> use metricsdb
Using database metricsdb
> select * from cpu limit 10 
name: cpu
time                cpu_usage          host          region
----                ---------          ----          ------
1573885718000000000 11.901733613473244 192.168.1.21  us-west
1573885718000000000 15.47121626535387  192.168.99.62 us-east
1573885718000000000 32.9044231821345   192.168.98.18 us-north
1573885718000000000 15.97303140480521  192.168.97.1  us-central
1573885718000000000 88.64886440612389  192.168.96.13 us-north
1573885718000000000 92.04980918501607  192.168.95.74 us-central
1573885718000000000 83.30404929547693  192.168.91.22 us-west
1573885718000000000 18.495673741297637 192.168.90.58 us-west
1573885718000000000 23.38901519689525  192.168.9.91  us-west
1573885718000000000 46.00933676790605  192.168.9.30  us-central

Leave a Reply

Your email address will not be published. Required fields are marked *