Hardening SSH

SSH created key-pair

More information on this topic at securing-your-server.
On your local machine, create a 4096-bit RSA key-pair,

$ ssh-keygen -b 4096

If you get a message to say that this already exists or to over write a file, then do not proceed, unless you want to create a new key-pair. If one exists, then it is probably being used to ssh into somewhere else. Don’t want to mess that up!!!
To check if it already exists,

$ cat ~/.ssh/id_rsa

Copy key to remote server

Next copy this key to the remote server,

$ ssh-copy-id <username>@<remote-server>

Now login to the remote server. You should not be asked for the password, since the key-pair now exists.

We do not want root logins. Open,

/etc/ssh/sshd_config and navigate to ‘#Authentication:’ and change to,

PermitLogin no

and also change to this, further down,

# Change to no to disable tunnelled clear test passwords
PasswordAuthentication no

Raspberry Pi Linux – Managing User Accounts

This information is available on the Rspberry Pi site here. I am just recapping for myself.

Change a user’s password

$ sudo passwd

Remove a user’s password

$ sudo passwd -d

Create a new user

$ sudo adduser

Remove a user,

$ sudo userdel -r

‘pi’ is a ‘sudoer’, which means this user can run as root when a command is preceded by ‘sudo’.
To switch to root,

$ sudo su

Add a user to ‘sudoers’,

$ sudo visudo

And add a line under ‘root ALL=(ALL:ALL) ALL’, eeplacing ‘root’ with e.g.,

ALL=(ALL:ALL) ALL

Raspberry Pi 3 Running noip2 systemd (part 9)

I find whenever the Pi has to reboot that I have to start noip2 each time with,

$ sudo noip2

This is exactly where systemd would be very useful. It would automatically restart the noip2 on boot or anytime it stops.

First we need to find where the noip2 binary is,

$ which noip2
/usr/local/bin/noip2

This is useful for finding the file that runs any command e.g

$ which ls
/bin/ls

Anyway, now we know that noip2 runs at /usr/local/bin/noip2. I would like to create a systemd service to run that binary file.

Create the file noip2.service at

/etc/systemd/system/noip2.service

[Unit]
Description=No-ip.com dynamic IP address updater
After=network.target
After=syslog.target

[Install]
WantedBy=multi-user.target
Alias=noip.service

[Service]
# Start main service
ExecStart=/usr/local/bin/noip2
Restart=always
Type=forking

I obtained this file from Nathan Giesbrecht’s GitHub page.

Enable the service, so that it will start on boot.

$ sudo systemctl enable noip2.service

Then start it,

$ sudo systemctl start noip2.service

Sometimes it is necessary to also,

$ systemctl daemon-reload

Check the status of this new service with,

$ systemctl status noip2.service

To test it, reboot your Pi.

$ sudo reboot

Check that the noip2 is running automatically with,

$ ps -aux|grep noip2

The second line is I presume the process to run this command, so ignore it. The first line shows this process is running.

Raspberry Pi 3 Express Application (part 8)

Check Node and npm are installed,

$ node --version
v7.4.0
$ npm --version
4.0.5

Install Express generator globally,

$ npm install express-generator -g

Check it,

$ express --version
4.14.0

Create a new directory inside /opt/nodeserver

$ mkdir tempGraph

This will create several folders as well as the package.json file with all dependencies. Next install these dependencies,

$ sudo npm install

This automatically locates package.json in the same directory as the command is run in and installs them.

Start express with,

$ npm start

open another terminal on the same system and type,

$ curl localhost:3000

You should see same HTML,

Install nodemon to automatically restart application should it be updated, which will happen frequently during development.

$ npm install -g nodemon

To start nodemon, first top the application and be sure to be in the applications directory, then,

$ nodemon

Nodemon is for development not production.

Raspberry Pi 3 Crontab (part 7)

To run a file every 5 minutes we will use crontab,

$ crontab -e

And add the following to the bottom of that file, which opens with the ‘nano’ editor.

# m h dom mon dow command
*/5 * * * * python /opt/nodeserver/temperature.py # At every 5th minute
1 */1 * * * python /opt/nodeserver/temperatureHourlyAverages.py # At minute 1 past every hour
2 * */1 * * python /opt/nodeserver/temperatureDailyAverages.py # At minute 2 on every day-of-month

This data can be graphed using d3.js The file and graph are available at shanespi.no-ip.biz if the server is running at the time, that is. Next I want to run this with the Express backend framework as a systemd service.

Systemd could be used to run the python files instead of the crontab.

Raspberry Pi 3 DS18B20 Temperature Sensor (part 6)

Open /boot/config.txt

$ sudo nano /boot/config.txt

and add

dtoverlay=w1-gpio

to the end of the file.

Reboot the system,

$ sudo reboot

Then run,

$ sudo modprobe w1-gpio
$ sudo modprobe w1-therm

The change directory /sys/bus/w1/devices/ and list its contents,

Change directory into 28-000006c87ee2 and then ‘cat w1_slave’.

The ‘t=16125’ this a temperature reading of 16.125 degrees celcius from the temperature sensor.

Check out Adafruit’s article on reading the DS18B20.

Using Python to read the temperature sensor and output the value to a file data.json

# /opt/nodeserver/temperature.py
# crontab -e runs this file every 5 minutes
# Reads temperature from sensor
# Reads /var/www/html/data/data/json file
# Appends a reading and removes earliest reading, only ever 13 readings allowed
# Dumps data back into /var/www/html/data/data,json

import os
from decimal import Decimal
import json
import datetime
import time

now = datetime.datetime.now()

# Open the file that we viewed earlier so that python can see what is in it. Replace the serial number as before.
tfile = open("/sys/bus/w1/devices/28-000006c87ee2/w1_slave")
# Read all of the text in the file.
text = tfile.read()
# Close the file now that the text has been read.
tfile.close()
# Split the text with new lines (\n) and select the second line.
secondline = text.split("\n")[1]
# Split the line into words, referring to the spaces, and select the 10th word (counting from 0).
temperaturedata = secondline.split(" ")[9]
# The first two characters are "t=", so get rid of those and convert the temperature from a string to a number.
temperature = Decimal(temperaturedata[2:])
# Put the decimal point in the right place and display it.
temp = temperature/1000

with open('/opt/nodeserver/data/data.json', 'r') as f:
data = json.load(f)

data[0]["fiveMinReadings"].append({"temp": str(temp), "date": str(now)})

while len(data[0]["fiveMinReadings"]) > 13:
del data[0]["fiveMinReadings"][0]

with open('/opt/nodeserver/data/data.json', 'w') as f:
json.dump(data, f)

temperatureDailyAverages.py calculates an average of the last 13 values from the temperature.py file, effectively finding an average for each hour. The result is appended to data.json

# /opt/nodeserver/temperatureHourlyAverages.py
# crontab -e runs this file every hour
# Reads /var/www/html/data/data/json file
# Calculates average reading from last hour
# Appends a reading and removes earliest reading, only ever 25 readings allowed
# Dumps data back into /var/www/html/data/data.json

import os
from decimal import Decimal
import json
import datetime
import time

now = datetime.datetime.now()
with open('/opt/nodeserver/data/data.json', 'r') as f:
data = json.load(f)

sum = 0;
for i in range(0,len(data[0]["fiveMinReadings"])):
sum = sum + float(data[0]["fiveMinReadings"][i]["temp"])

average = sum/len(data[0]["fiveMinReadings"]) #this is the average reading from the previous 13 readings taken every 5 minutes
temp = round(average, 2)

data[1]["hourlyAverages"].append({"date": str(now),"temp": str(temp)})

while len(data[1]["hourlyAverages"]) > 25:
del data[1]["hourlyAverages"][0]

with open('/opt/nodeserver/html/data/data.json', 'w') as f:
json.dump(data, f)

Similarly the temperatureHourlyAverages.py

# /opt/nodeserver/temperatureHourlyAverages.py
# crontab -e runs this file every day at midnight 00:00
# Reads /var/www/html/data/data/json file
# Calculates average reading from last 24 hours
# Appends a reading and removes earliest reading, only ever 30 readings allowed
# Dumps data back into /var/www/html/data/data.json

import os
from decimal import Decimal
import json
import datetime
import time

now = datetime.datetime.now()
with open('/opt/nodeserver/data/data.json', 'r') as f:
data = json.load(f)

sum = 0;
for i in range(0,len(data[1]["hourlyAverages"])):
sum = sum + float(data[1]["hourlyAverages"][i]["temp"])

average = sum/len(data[1]["hourlyAverages"]) #this is the average reading from the previous 30 readings taken every hour
temp = round(average, 2)

data[2]["dailyAverages"].append({"date": str(now),"temp": str(temp)})

while len(data[2]["dailyAverages"]) > 30:
del data[2]["dailyAverages"][0]

with open('/opt/nodeserver/data/data.json', 'w') as f:
json.dump(data, f)

Raspberry Pi 3 Node Server & systemd (part 5)

Run a simple Node server. Create the file and path,

/opt/nodeserver/server.js

const http = require('http');
const hostname = '0.0.0.0'; // listen on all ports
const port = 8080;
http.createServer((req, res) => {
res.writeHead(200, { 'Content-Type': 'text/plain' });
res.end('Hello people, I am a server \n' + new Date().toISOString());
}).listen(port, hostname, () => {
console.log('Server running at http://' + hostname + ':' + port);
});

Run this,

$ node server.js

But this will stop running if you ctrl-c. Therefore we will use Systemd to create a service which will run this server continuously and also start it again in the case of any interruption.

Change directory to

$ cd /etc/systemd/system

Create the file nodeserver.service

$ sudo nano nodeserver.service

and add the following [Unit], [Service] and [Install] as follows,

[Unit]
Description=Node.js Example Server
#Requires=After=mysql.service # Requires the mysql service to run first

[Service]

ExecStart=/usr/bin/node /opt/nodeserver/server.js
Restart=always
RestartSec=10 # Restart service after 10 seconds if node service crashes
StandardOutput=syslog # Output to syslog
StandardError=syslog # Output to syslog
SyslogIdentifier=nodejs-example
User=pi
Group=root
Environment=NODE_ENV=production PORT=8080

[Install]
WantedBy=multi-user.target

You may need to change to ‘User’ and ‘Group’. Make sure the PORT=8080 is the same port as in server.js

Next we need to ‘enable’ and ‘start’ the service

$ sudo systemctl enable nodeserver.service
$ sudo systemctl start nodeserver.service

You can also use ‘restart’, ‘stop’ and ‘status’ to check that the service is running.

$ sudo systemctl restart nodeserver.service
$ sudo systemctl stop nodeserver.service
$ sudo systemctl status nodeserver.service

You will need to reload the daemon with

$ sudo systemctl daemon-reload

After ‘reload’ you will need ‘restart’ the service.

$ sudo systemctl restart nodeserver.service

Most of the information here is from the excellent tutorial at nodejs service with systemd
Another good source,
Deploying nodejs applications with systemd

You can check this in your browser with,

curl http://0.0.0.0:8080

If you’ve set up port-forwarding and noip.com you’ll be able to go to any browser on the internet and see the output.

Raspberry Pi 3 Static IP Address & No-ip setup (part 3)

To set up a static IP address on the Pi 3 which will remain constant inside you LAN open /etc/dhcpcd.conf

$ sudo nano dhcpcd.conf

Add to the bottom of this file,

interface eth0

static ip_address=192.168.1.30
static routers=192.168.1.1
static domain_name_servers=192.168.1.1

interface wlan0

static ip_address=192.168.1.30
static routers=192.168.1.1
static domain_name_servers=192.168.1.1

Here 192.1.68.1.30 is the static ip address I am creating.
192.1.68.1.1 is the gateway address, the address of my router from inside the network.

You will need to access your router’s settings and reserve the ip address 192.1.68.1.30 or whatever you chose, so that no other device on your network will be allocated this address.

Your router external address, that is the ip with which it is identified on the internet is usually dynamic. You can fix this using ‘no-ip’, this will allow you set it to a name instead of having to find the ever changing ip address.

Sign up for a free account at no-ip.com The free account needs to be renewed every 30 days. But this is a excellent and very useful service.

The instructions for setting up noip2 on a Raspberry Pi 3 can be found on the no-ip website.

Raspberry Pi 3 SD card image setup (part 1)

Download the latest Raspbian

Raspbian Jessie with Pixel

Unzip the .zip file

$ unzip 2016-11-25-raspbian-jessie.zip

This will create a new image file,

2016-11-25-raspbian-jessie.img

Look at the mounted filesystems before you put in the microSD card in it’s adapter,

$ df -h

Place the microSD card in the SD adapter and slot it into your computer. Check the name for this on your system,

$ df -h

Here we see the new partitions on the SD card, which were not present the first time we ran ‘df -h’. These are ‘mmcblk0p3′,’mmcblk0p5’ and ‘mmcblk0p6’. The card itself is ‘mmcblk0’. If this is a new card you won’t see these with ‘df -h’ In that case change directory to /dev

$ /dev
$ ls

and you will see the mmcblk0** partitions.

If they do not show up with ‘df -h’ then they are not mounted. You do not want them mounted when you copy the image to the SD card. It is important to unmount this partitions with umount,

Now we are ready to put the .img file on the microSD card at ‘mmcblk0’. ‘mmcblk0’ with no partition after it references the whole card, which is what we want.

We will use ‘dd’ to copy the image onto the SD card.

Then remove the card and slot it into your Pi 3.

I don’t know why, but I had to repeat this process. The card did work in the Pi3 but the wifi wouldn’t connect or the ethernet connection.

Setting up gh-pages

Login to github.com and create a new reporitory. Copy the link for this repository. It will look like this,

https://github.com/<user-name>/<repo-name>.git

On your local machine create a new directory,

$ mkdir testGHpages

cd into it

$ cd testGHpages

git init this directory

$ git init

Add a simple index.html file to this directory.
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
</head>
<body>
<span>Hello world!</span>
</body>
</html>

Add all directory contents,
$ git add .

Commit directory contents,
$ git commit -m "initial commit"

Add your the repositories which you made to this directory.
$ git remote add origin https://github.com/<username>/<repo-name>.git

Push the contents of this directory to the repository on github.
$ git push origin master
Your username and password will be required.

Create a gh-pages branch,
$ git checkout -b gh-pages

Push your the contents to this branch,
$ git push origin gh-pages

You should be able to see your page at,
https://<user-name>.github.io/<repository-name>

Not necessary but you can merge your main branch with the gh-pages branch with,
$ git merge master

To go back to the main branch,
$ git checkout master

D3.js Basic json Graph

Basic example of graphing using D3.js
Useful links used for this post:
http://alignedleft.com/tutorials/d3/axes
https://square.github.io/intro-to-d3/

<!DOCTYPE html>
<html lang='en'>
  <head>
    <meta charset='utf-8'>
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.1.1/jquery.min.js" type="text/javascript"></script> 
    <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/d3/3.5.17/d3.min.js"></script>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/crossfilter/1.3.12/crossfilter.min.js" type="text/javascript"></script>
    <script src="http://cdnjs.cloudflare.com/ajax/libs/dc/1.7.5/dc.min.js" type="text/javascript"></script>
    <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js" type="text/javascript"></script>
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css">
    <link rel="stylesheet" href="css/tempGraph.css">
    <link href="http://cdnjs.cloudflare.com/ajax/libs/dc/1.7.5/dc.min.css" rel="stylesheet" type="text/css">
    <script src="https://d3js.org/d3-array.v1.min.js"></script>
    <script src="https://d3js.org/d3-collection.v1.min.js"></script>
    <script src="https://d3js.org/d3-color.v1.min.js"></script>
    <script src="https://d3js.org/d3-format.v1.min.js"></script>
    <script src="https://d3js.org/d3-interpolate.v1.min.js"></script>
    <script src="https://d3js.org/d3-time.v1.min.js"></script>
    <script src="https://d3js.org/d3-time-format.v2.min.js"></script>
    <script src="https://d3js.org/d3-scale.v1.min.js"></script>
    
  </head>
  
  <body>
    <div class='container' id='main-container'>
      <div class='content'>
	<div class='container' style='font: 10px sans-serif;'>
	  <h3></h3>
	  <div class='row-fluid'>
	    <div class='remaining-graphs span8'>
	      <div class='row-fluid'>
                <div class='pie-graph span8' id='dc-line-chart'>
 
		</div>
	      </div>
	    </div>
	  </div>
	</div>
      </div>
    </div>
    <script>
      var data = [
          {xVal : 1, yVal : 1},
          {xVal : 2, yVal : 4},
          {xVal : 3, yVal : 2},
          {xVal : 4, yVal : 3}
      ];
      var padding = 25;
      var w = 500;
      var h = 150;
      
      var xScale = d3.scaleLinear()
          .domain([0,d3.max(data, function(d) {return d.xVal})])
          .range([0, 200]);

      var yScale = d3.scaleLinear()
          .domain([0,d3.max(data, function(d) {return d.yVal})])
          .range([100, 0]);

      //var xAxis = d3.axisBottom(x) // d3 v.3 only
      var xAxis =  d3.svg.axis(xScale) // d3 v.4
          .ticks(4) // specify the number of ticks
          .scale(xScale)
          .orient("bottom");

      var yAxis = d3.svg.axis(yScale)
          .scale(yScale)
          .orient("left")
          .ticks(7);
      
      var svg = d3.select('#dc-line-chart')
          .append('svg')        // create an <svg> element
          .attr('width', w) // set its dimensions
          .attr('height', h);

        svg.append("g")
        .attr("class", "axis")
        .attr("transform", "translate(" + (padding) + "," + padding + ")")
        .call(yAxis);
	
	svg.append('g')            // create a <g> element
          .attr('class', 'axis')   // specify classes
	  .attr("transform", "translate(" + padding + "," + (h - padding) + ")")
          .call(xAxis);            // let the axis do its thing
	  
      var svg = d3.select('svg');
      svg.size();

      var rects = svg.selectAll('rect')
          .data(data);
          rects.size();
	
      var newRects = rects.enter();
      
      newRects.append('rect')
          .attr('x', function(d, i) {
              return xScale(d.xVal);
          })
          .attr('y', function(d, i) {
              return yScale(d.yVal);
	  })
	  .attr("transform","translate(" + (padding -5) + "," + (padding - 5) + ")")
          .attr('height', 10)
          .attr('width', 10);
      
    </script>
  </body>
</html>

tempGraph.css

.axis path,
.axis line {
    fill: none;
    stroke: black;
    shape-rendering: crispEdges;
}

.axis text {
    font-family: sans-serif;
    font-size: 11px;
}

Part 7: Django Login, Logout & Registration

Login form:
Adapted from theĀ login tutorial at techand startup.
Create a new app called ‘accounts’

$ python manage.py startapp accounts

Add ‘accounts’ to settings.py

INSTALLED_APPS = [
'accounts',
'django.contrib.admin',
'django.contrib.auth',
.............

Go to crudProject/views.py and add to the top of the file,

from django.shortcuts import render_to_response

Below that add,

def loggedin(request):
return render_to_response('registration/loggedin.html')

If you want to show their username when they login then call their username in the view. Change the loggedin function to:

def loggedin(request):
return render_to_response('registration/loggedin.html',
{'username': request.user.username})

Add the following urls to crudProject/urls.py

 # Auth-related URLs:
url(r'^accounts/login/$', 'django.contrib.auth.views.login', name='login'),
url(r'^accounts/logout/$', 'django.contrib.auth.views.logout', name='logout'),
url(r'^accounts/loggedin/$', 'crudProject.views.loggedin', name='loggedin'),

To the bottom of crudProject/settings.py add,

LOGIN_REDIRECT_URL = '/accounts/loggedin/'

Add login and logout buttons to index.html

<a href="/accounts/logout">logout</a>
<a href="/accounts/login">login</a>

Create a ‘registration’ directory in templates and add ‘login.html’

{% extends "index.html" %}
{% block title %}Log In{% endblock %}
{% block content %}

<form method="post" action="{% url 'django.contrib.auth.views.login' %}">
{% csrf_token %}
<table>
{{ form.as_table }}
</table>

<input type="submit" value="login" />
</form>

{% endblock %}

In templates/registration/loggedin.html put,

{% extends "index.html" %}
{% block title %}Logged In{% endblock %}
{% block content %}

<h1>Welcome {{username}}</h1>
<p>Thank you for logging in.</p>
<p><a href="/accounts/logout/">Logout</a></p>

{% endblock %}

The create templates/registration/logged_out.html

{% extends "index.html" %}
{% block title %}Logged Out{% endblock %}
{% block content %}

<h2>Logged out!</h2>
<p><a href="/accounts/login/">Log back in</a></p>

{% endblock %}

User Registration:
Add to crudProject/views.py

from django.shortcuts import render_to_response
from django.http import HttpResponseRedirect
from django.contrib.auth.forms import UserCreationForm
from django.core.context_processors import csrf

def register(request):
if request.method == 'POST':
form = UserCreationForm(request.POST)
if form.is_valid():
form.save()
return HttpResponseRedirect('/accounts/register/complete')

else:
form = UserCreationForm()
token = {}
token.update(csrf(request))
token['form'] = form

return render_to_response('registration/registration_form.html', token)

def registration_complete(request):
return render_to_response('registration/registration_complete.html')

Add to crudProject/urls.py

# Registration URLs
url(r'^accounts/register/$', 'crudProject.views.register', name='register'),
url(r'^accounts/register/complete/$', 'crudProject.views.registration_complete', name='registration_complete'),
...........................

Python Dictionary into HTML Template

The following Python code will find instances of {body} in a file template.html and replace it with a string. Then a new file index.html is created with this change.

generate.py

with open('template.html','r') as template_file:
    template = template_file.read()

tokens = {'title': 'My Title in a Dictionary', 'header':'Stuff for heading', 'body': 'The body stuff', 'footer': 'This is the footer'}
content = template

for token in tokens:
    content = content.replace('{' + token + '}', tokens[token])

with open('index.html', 'w') as content_file:
     content_file.write(content)

index.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>{title}</title>
</head>
    <div><p>{header}</p></div>
    <div><p>{body}</p></div>
    <div><p>{footer}</p></div>
</html>