Raspberry Pi Linux – Managing User Accounts

This information is available on the Rspberry Pi site here. I am just recapping for myself.

Change a user’s password

$ sudo passwd

Remove a user’s password

$ sudo passwd -d

Create a new user

$ sudo adduser

Remove a user,

$ sudo userdel -r

‘pi’ is a ‘sudoer’, which means this user can run as root when a command is preceded by ‘sudo’.
To switch to root,

$ sudo su

Add a user to ‘sudoers’,

$ sudo visudo

And add a line under ‘root ALL=(ALL:ALL) ALL’, eeplacing ‘root’ with e.g.,

ALL=(ALL:ALL) ALL

Raspberry Pi 3 Running noip2 systemd (part 9)

I find whenever the Pi has to reboot that I have to start noip2 each time with,

$ sudo noip2

This is exactly where systemd would be very useful. It would automatically restart the noip2 on boot or anytime it stops.

First we need to find where the noip2 binary is,

$ which noip2
/usr/local/bin/noip2

This is useful for finding the file that runs any command e.g

$ which ls
/bin/ls

Anyway, now we know that noip2 runs at /usr/local/bin/noip2. I would like to create a systemd service to run that binary file.

Create the file noip2.service at

/etc/systemd/system/noip2.service

[Unit]
Description=No-ip.com dynamic IP address updater
After=network.target
After=syslog.target

[Install]
WantedBy=multi-user.target
Alias=noip.service

[Service]
# Start main service
ExecStart=/usr/local/bin/noip2
Restart=always
Type=forking

I obtained this file from Nathan Giesbrecht’s GitHub page.

Enable the service, so that it will start on boot.

$ sudo systemctl enable noip2.service

Then start it,

$ sudo systemctl start noip2.service

Sometimes it is necessary to also,

$ systemctl daemon-reload

Check the status of this new service with,

$ systemctl status noip2.service

To test it, reboot your Pi.

$ sudo reboot

Check that the noip2 is running automatically with,

$ ps -aux|grep noip2

The second line is I presume the process to run this command, so ignore it. The first line shows this process is running.

Raspberry Pi 3 Express Application (part 8)

Check Node and npm are installed,

$ node --version
v7.4.0
$ npm --version
4.0.5

Install Express generator globally,

$ npm install express-generator -g

Check it,

$ express --version
4.14.0

Create a new directory inside /opt/nodeserver

$ mkdir tempGraph

This will create several folders as well as the package.json file with all dependencies. Next install these dependencies,

$ sudo npm install

This automatically locates package.json in the same directory as the command is run in and installs them.

Start express with,

$ npm start

open another terminal on the same system and type,

$ curl localhost:3000

You should see same HTML,

Install nodemon to automatically restart application should it be updated, which will happen frequently during development.

$ npm install -g nodemon

To start nodemon, first top the application and be sure to be in the applications directory, then,

$ nodemon

Nodemon is for development not production.

Raspberry Pi 3 Crontab (part 7)

To run a file every 5 minutes we will use crontab,

$ crontab -e

And add the following to the bottom of that file, which opens with the ‘nano’ editor.

# m h dom mon dow command
*/5 * * * * python /opt/nodeserver/temperature.py # At every 5th minute
1 */1 * * * python /opt/nodeserver/temperatureHourlyAverages.py # At minute 1 past every hour
2 * */1 * * python /opt/nodeserver/temperatureDailyAverages.py # At minute 2 on every day-of-month

This data can be graphed using d3.js The file and graph are available at shanespi.no-ip.biz if the server is running at the time, that is. Next I want to run this with the Express backend framework as a systemd service.

Systemd could be used to run the python files instead of the crontab.

Raspberry Pi 3 DS18B20 Temperature Sensor (part 6)

Open /boot/config.txt

$ sudo nano /boot/config.txt

and add

dtoverlay=w1-gpio

to the end of the file.

Reboot the system,

$ sudo reboot

Then run,

$ sudo modprobe w1-gpio
$ sudo modprobe w1-therm

The change directory /sys/bus/w1/devices/ and list its contents,

Change directory into 28-000006c87ee2 and then ‘cat w1_slave’.

The ‘t=16125’ this a temperature reading of 16.125 degrees celcius from the temperature sensor.

Check out Adafruit’s article on reading the DS18B20.

Using Python to read the temperature sensor and output the value to a file data.json

# /opt/nodeserver/temperature.py
# crontab -e runs this file every 5 minutes
# Reads temperature from sensor
# Reads /var/www/html/data/data/json file
# Appends a reading and removes earliest reading, only ever 13 readings allowed
# Dumps data back into /var/www/html/data/data,json

import os
from decimal import Decimal
import json
import datetime
import time

now = datetime.datetime.now()

# Open the file that we viewed earlier so that python can see what is in it. Replace the serial number as before.
tfile = open("/sys/bus/w1/devices/28-000006c87ee2/w1_slave")
# Read all of the text in the file.
text = tfile.read()
# Close the file now that the text has been read.
tfile.close()
# Split the text with new lines (\n) and select the second line.
secondline = text.split("\n")[1]
# Split the line into words, referring to the spaces, and select the 10th word (counting from 0).
temperaturedata = secondline.split(" ")[9]
# The first two characters are "t=", so get rid of those and convert the temperature from a string to a number.
temperature = Decimal(temperaturedata[2:])
# Put the decimal point in the right place and display it.
temp = temperature/1000

with open('/opt/nodeserver/data/data.json', 'r') as f:
data = json.load(f)

data[0]["fiveMinReadings"].append({"temp": str(temp), "date": str(now)})

while len(data[0]["fiveMinReadings"]) > 13:
del data[0]["fiveMinReadings"][0]

with open('/opt/nodeserver/data/data.json', 'w') as f:
json.dump(data, f)

temperatureDailyAverages.py calculates an average of the last 13 values from the temperature.py file, effectively finding an average for each hour. The result is appended to data.json

# /opt/nodeserver/temperatureHourlyAverages.py
# crontab -e runs this file every hour
# Reads /var/www/html/data/data/json file
# Calculates average reading from last hour
# Appends a reading and removes earliest reading, only ever 25 readings allowed
# Dumps data back into /var/www/html/data/data.json

import os
from decimal import Decimal
import json
import datetime
import time

now = datetime.datetime.now()
with open('/opt/nodeserver/data/data.json', 'r') as f:
data = json.load(f)

sum = 0;
for i in range(0,len(data[0]["fiveMinReadings"])):
sum = sum + float(data[0]["fiveMinReadings"][i]["temp"])

average = sum/len(data[0]["fiveMinReadings"]) #this is the average reading from the previous 13 readings taken every 5 minutes
temp = round(average, 2)

data[1]["hourlyAverages"].append({"date": str(now),"temp": str(temp)})

while len(data[1]["hourlyAverages"]) > 25:
del data[1]["hourlyAverages"][0]

with open('/opt/nodeserver/html/data/data.json', 'w') as f:
json.dump(data, f)

Similarly the temperatureHourlyAverages.py

# /opt/nodeserver/temperatureHourlyAverages.py
# crontab -e runs this file every day at midnight 00:00
# Reads /var/www/html/data/data/json file
# Calculates average reading from last 24 hours
# Appends a reading and removes earliest reading, only ever 30 readings allowed
# Dumps data back into /var/www/html/data/data.json

import os
from decimal import Decimal
import json
import datetime
import time

now = datetime.datetime.now()
with open('/opt/nodeserver/data/data.json', 'r') as f:
data = json.load(f)

sum = 0;
for i in range(0,len(data[1]["hourlyAverages"])):
sum = sum + float(data[1]["hourlyAverages"][i]["temp"])

average = sum/len(data[1]["hourlyAverages"]) #this is the average reading from the previous 30 readings taken every hour
temp = round(average, 2)

data[2]["dailyAverages"].append({"date": str(now),"temp": str(temp)})

while len(data[2]["dailyAverages"]) > 30:
del data[2]["dailyAverages"][0]

with open('/opt/nodeserver/data/data.json', 'w') as f:
json.dump(data, f)

Raspberry Pi 3 Node Server & systemd (part 5)

Run a simple Node server. Create the file and path,

/opt/nodeserver/server.js

const http = require('http');
const hostname = '0.0.0.0'; // listen on all ports
const port = 8080;
http.createServer((req, res) => {
res.writeHead(200, { 'Content-Type': 'text/plain' });
res.end('Hello people, I am a server \n' + new Date().toISOString());
}).listen(port, hostname, () => {
console.log('Server running at http://' + hostname + ':' + port);
});

Run this,

$ node server.js

But this will stop running if you ctrl-c. Therefore we will use Systemd to create a service which will run this server continuously and also start it again in the case of any interruption.

Change directory to

$ cd /etc/systemd/system

Create the file nodeserver.service

$ sudo nano nodeserver.service

and add the following [Unit], [Service] and [Install] as follows,

[Unit]
Description=Node.js Example Server
#Requires=After=mysql.service # Requires the mysql service to run first

[Service]

ExecStart=/usr/bin/node /opt/nodeserver/server.js
Restart=always
RestartSec=10 # Restart service after 10 seconds if node service crashes
StandardOutput=syslog # Output to syslog
StandardError=syslog # Output to syslog
SyslogIdentifier=nodejs-example
User=pi
Group=root
Environment=NODE_ENV=production PORT=8080

[Install]
WantedBy=multi-user.target

You may need to change to ‘User’ and ‘Group’. Make sure the PORT=8080 is the same port as in server.js

Next we need to ‘enable’ and ‘start’ the service

$ sudo systemctl enable nodeserver.service
$ sudo systemctl start nodeserver.service

You can also use ‘restart’, ‘stop’ and ‘status’ to check that the service is running.

$ sudo systemctl restart nodeserver.service
$ sudo systemctl stop nodeserver.service
$ sudo systemctl status nodeserver.service

You will need to reload the daemon with

$ sudo systemctl daemon-reload

After ‘reload’ you will need ‘restart’ the service.

$ sudo systemctl restart nodeserver.service

Most of the information here is from the excellent tutorial at nodejs service with systemd
Another good source,
Deploying nodejs applications with systemd

You can check this in your browser with,

curl http://0.0.0.0:8080

If you’ve set up port-forwarding and noip.com you’ll be able to go to any browser on the internet and see the output.

Raspberry Pi 3 Node.js setup (part 4)

This is from the excellent site thisdavej.com

Check the ARM architecture of your chip,

$ uname -m

It should be armv7l

Update the Debian package repository to include NodeSource

$ curl -sL https://deb.nodesource.com/setup_7.x | sudo -E bash -

Install Node.js

$ sudo apt-get install nodejs

Check it is installed with

$ nodejs --version

Run node with,

$ node
> 1+2
3

Raspberry Pi 3 Static IP Address & No-ip setup (part 3)

To set up a static IP address on the Pi 3 which will remain constant inside you LAN open /etc/dhcpcd.conf

$ sudo nano dhcpcd.conf

Add to the bottom of this file,

interface eth0

static ip_address=192.168.1.30
static routers=192.168.1.1
static domain_name_servers=192.168.1.1

interface wlan0

static ip_address=192.168.1.30
static routers=192.168.1.1
static domain_name_servers=192.168.1.1

Here 192.1.68.1.30 is the static ip address I am creating.
192.1.68.1.1 is the gateway address, the address of my router from inside the network.

You will need to access your router’s settings and reserve the ip address 192.1.68.1.30 or whatever you chose, so that no other device on your network will be allocated this address.

Your router external address, that is the ip with which it is identified on the internet is usually dynamic. You can fix this using ‘no-ip’, this will allow you set it to a name instead of having to find the ever changing ip address.

Sign up for a free account at no-ip.com The free account needs to be renewed every 30 days. But this is a excellent and very useful service.

The instructions for setting up noip2 on a Raspberry Pi 3 can be found on the no-ip website.

Raspberry Pi 3 Wifi setup (part 2)

First slot in the SD card with the ethernet cable plugged in and USB mouse and keyboard and also a monitor usually connected by a HDMI connection.

Ethernet should be working. Check by pinging a site in the terminal,

$ ping duckduckgo.com

If this doesn’t work you won’t be able to upgrade the system.

Update and upgrade with the Debian package manager, ‘apt-get’

$ sudo apt-get update
$ sudo apt-get upgrade

Then update the Linux distribution,

$ sudo apt-get dist-update

Change directory to the wpa_supplicant.cong file to set the wifi network and password.

$ cd /etc/wpa_supplicant

Then open the file wpa_supplicant.conf

$ sudo nano wpa_supplicant.conf

At the end of the file add,

network={
ssid="your-network-name"
psk="your-wifi-passord"
}

Ctrl-X and Y will save this file. Ctrl-o will save without closing.

At this stage I went to the Pi icon in the top left of the screen and ‘Preferences’ > ‘Raspberry Pi Configuration’

Go to ‘Change Password’. The current password is ‘raspberry’ and choose a new one.
Then under the interfaces tab, it is a good idea to at least enable SSH. Then under the ‘Localisation’ tab set the ‘Set Locale’, ‘Set Timezone’ and ‘Set WiFi Country’.

Next reboot,

$ sudo reboot

When teh Pi boots again change your ip address on the network with,

$ ifconfig

You should see three blocks, one labelled ‘eth0’ for ethernet, ‘lo’ fpr loopbacka nd ‘wlan0’ for the wifi connection. The wlan0 block should have an ip address such as 192.168.1.11 in the line,

inet addr: 192.168.1.11

This is the Pi’s ip address on your local network. Take note of it.

Go to another computer on your network and open a terminal or command line interface (CLI). Type,

$ ssh pi@192.168.1.11

You will be asked for a password and hopefully you will be connected to the pi’s command line and so not need to monitor. This is called a headless connection.

Raspberry Pi 3 SD card image setup (part 1)

Download the latest Raspbian

Raspbian Jessie with Pixel

Unzip the .zip file

$ unzip 2016-11-25-raspbian-jessie.zip

This will create a new image file,

2016-11-25-raspbian-jessie.img

Look at the mounted filesystems before you put in the microSD card in it’s adapter,

$ df -h

Place the microSD card in the SD adapter and slot it into your computer. Check the name for this on your system,

$ df -h

Here we see the new partitions on the SD card, which were not present the first time we ran ‘df -h’. These are ‘mmcblk0p3′,’mmcblk0p5’ and ‘mmcblk0p6’. The card itself is ‘mmcblk0’. If this is a new card you won’t see these with ‘df -h’ In that case change directory to /dev

$ /dev
$ ls

and you will see the mmcblk0** partitions.

If they do not show up with ‘df -h’ then they are not mounted. You do not want them mounted when you copy the image to the SD card. It is important to unmount this partitions with umount,

Now we are ready to put the .img file on the microSD card at ‘mmcblk0’. ‘mmcblk0’ with no partition after it references the whole card, which is what we want.

We will use ‘dd’ to copy the image onto the SD card.

Then remove the card and slot it into your Pi 3.

I don’t know why, but I had to repeat this process. The card did work in the Pi3 but the wifi wouldn’t connect or the ethernet connection.

Setting up gh-pages

Login to github.com and create a new reporitory. Copy the link for this repository. It will look like this,

https://github.com/<user-name>/<repo-name>.git

On your local machine create a new directory,

$ mkdir testGHpages

cd into it

$ cd testGHpages

git init this directory

$ git init

Add a simple index.html file to this directory.
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
</head>
<body>
<span>Hello world!</span>
</body>
</html>

Add all directory contents,
$ git add .

Commit directory contents,
$ git commit -m "initial commit"

Add your the repositories which you made to this directory.
$ git remote add origin https://github.com/<username>/<repo-name>.git

Push the contents of this directory to the repository on github.
$ git push origin master
Your username and password will be required.

Create a gh-pages branch,
$ git checkout -b gh-pages

Push your the contents to this branch,
$ git push origin gh-pages

You should be able to see your page at,
https://<user-name>.github.io/<repository-name>

Not necessary but you can merge your main branch with the gh-pages branch with,
$ git merge master

To go back to the main branch,
$ git checkout master

D3.js Basic json Graph

Basic example of graphing using D3.js version 3
Useful links used for this post:
http://alignedleft.com/tutorials/d3/axes
https://square.github.io/intro-to-d3/

<!DOCTYPE html>
<html lang='en'>
  <head>
    <meta charset='utf-8'>
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.1.1/jquery.min.js" type="text/javascript"></script> 
    <script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/d3/3.5.17/d3.min.js"></script>
    <script src="https://cdnjs.cloudflare.com/ajax/libs/crossfilter/1.3.12/crossfilter.min.js" type="text/javascript"></script>
    <script src="http://cdnjs.cloudflare.com/ajax/libs/dc/1.7.5/dc.min.js" type="text/javascript"></script>
    <script src="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/js/bootstrap.min.js" type="text/javascript"></script>
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css">
    <link rel="stylesheet" href="css/tempGraph.css">
    <link href="http://cdnjs.cloudflare.com/ajax/libs/dc/1.7.5/dc.min.css" rel="stylesheet" type="text/css">
    <script src="https://d3js.org/d3-array.v1.min.js"></script>
    <script src="https://d3js.org/d3-collection.v1.min.js"></script>
    <script src="https://d3js.org/d3-color.v1.min.js"></script>
    <script src="https://d3js.org/d3-format.v1.min.js"></script>
    <script src="https://d3js.org/d3-interpolate.v1.min.js"></script>
    <script src="https://d3js.org/d3-time.v1.min.js"></script>
    <script src="https://d3js.org/d3-time-format.v2.min.js"></script>
    <script src="https://d3js.org/d3-scale.v1.min.js"></script>
    
  </head>
  
  <body>
    <div class='container' id='main-container'>
      <div class='content'>
	<div class='container' style='font: 10px sans-serif;'>
	  <h3></h3>
	  <div class='row-fluid'>
	    <div class='remaining-graphs span8'>
	      <div class='row-fluid'>
                <div class='pie-graph span8' id='dc-line-chart'>
 
		</div>
	      </div>
	    </div>
	  </div>
	</div>
      </div>
    </div>
    <script>
      var data = [
          {xVal : 1, yVal : 1},
          {xVal : 2, yVal : 4},
          {xVal : 3, yVal : 2},
          {xVal : 4, yVal : 3}
      ];
      var padding = 25;
      var w = 500;
      var h = 150;
      
      var xScale = d3.scaleLinear()
          .domain([0,d3.max(data, function(d) {return d.xVal})])
          .range([0, 200]);

      var yScale = d3.scaleLinear()
          .domain([0,d3.max(data, function(d) {return d.yVal})])
          .range([100, 0]);

      //var xAxis = d3.axisBottom(x) // d3 v.3 only
      var xAxis =  d3.svg.axis(xScale) // d3 v.4
          .ticks(4) // specify the number of ticks
          .scale(xScale)
          .orient("bottom");

      var yAxis = d3.svg.axis(yScale)
          .scale(yScale)
          .orient("left")
          .ticks(7);
      
      var svg = d3.select('#dc-line-chart')
          .append('svg')        // create an <svg> element
          .attr('width', w) // set its dimensions
          .attr('height', h);

        svg.append("g")
        .attr("class", "axis")
        .attr("transform", "translate(" + (padding) + "," + padding + ")")
        .call(yAxis);
	
	svg.append('g')            // create a <g> element
          .attr('class', 'axis')   // specify classes
	  .attr("transform", "translate(" + padding + "," + (h - padding) + ")")
          .call(xAxis);            // let the axis do its thing
	  
      var svg = d3.select('svg');
      svg.size();

      var rects = svg.selectAll('rect')
          .data(data);
          rects.size();
	
      var newRects = rects.enter();
      
      newRects.append('rect')
          .attr('x', function(d, i) {
              return xScale(d.xVal);
          })
          .attr('y', function(d, i) {
              return yScale(d.yVal);
	  })
	  .attr("transform","translate(" + (padding -5) + "," + (padding - 5) + ")")
          .attr('height', 10)
          .attr('width', 10);
      
    </script>
  </body>
</html>

tempGraph.css

.axis path,
.axis line {
    fill: none;
    stroke: black;
    shape-rendering: crispEdges;
}

.axis text {
    font-family: sans-serif;
    font-size: 11px;
}

Basics of MongoDB Shell

With MongoDB installed, (see Install Mongodb on Fedora 23)

get the server running with,

$ mongod

and remember this will hang, so open a new tab in your terminal to continue. In fedora this will be

Ctrl+shift+T

In that new terminal,

$ sudo mongo

Let us see what if any databases exist on the server,

 > show dbs

Say I have a database called ‘mydb’. Now if we want to use that databse we must use,

 > use mydb

To see what ‘collections’ or ‘tables’ exist in this database use,

 > show collections

If there is a collection called ‘movie’ and we’d like to look at it’s contents use,

 > db.movie.find()

To create a new database just start using it,

 > use newDB

Django: Multiple Pagination in a single Template

Assuming you would like to display two models in the same template each with their own pagination, here’s how to do it.

views.py

def myview():
        Model_one = Model.objects.all()
        paginator = Paginator(Model_one, 6)
        page = request.GET.get('page1')
        try:
            Model_one = paginator.page(page)
        except PageNotAnInteger:
            Model_one = paginator.page(1)
        except EmptyPage:
            Model_one = paginator.page(paginator.num_pages)

        Model_two = Model_other.objects.all()
        paginator = Paginator(Model_two, 6)
        page = request.GET.get('page2')
        try:
            Model_two = paginator.page(page)
        except PageNotAnInteger:
            Model_two = paginator.page(1)
        except EmptyPage:
            Model_two = paginator.page(paginator.num_pages)

        context = {'Model_one': Model_one, 'Model_two': Model_two}
        return render(request, 'template.html', context)

The important thing above is the ‘page1’ and ‘page2’.

In the template,

{% if model_one %}
          <div class="col-md-12 well">
            {% for item in model_one %}
            ..... iterates through model_one.....
            {% endfor %}
            <span class="step-links pagination">
                {% if model_one.has_previous %}
                    <a href="?page1={{ model_one.previous_page_number }}"> previous </a>
                {% endif %}
                <span class="current">
                    Page {{ model_one.number }} of {{ model_one.paginator.num_pages }}
                </span>
                {% if model_one.has_next %}
                    <a href="?page1={{ model_one.next_page_number }}"> next </a>
                {% endif %}
            </span>
          </div>
          {% endif %}
          {% if model_two %}
          <div class="col-md-12 well">
            {% for item in model_two %}
            ..... iterates through model_two.....
            {% endfor %}
            <span class="step-links pagination">
                {% if model_two.has_previous %}
                    <a href="?page2={{ model_two.previous_page_number }}"> previous 
                {% endif %}
                <span class="current">
                    Page {{ model_two.number }} of {{ model_two.paginator.num_pages }}
                </span>
                {% if model_two.has_next %}
                    <a href="?page2={{ model_two.next_page_number }}"> next </a>
                {% endif %}
            </span>
          </div>
          {% endif %}

Again using ‘page1’ and ‘page2’ to distinguish the pagination for each model.

Django-allauth Installation

There is a useful django-allauth video tutorial at Create third party Facebook login in Django. The document below simply follows that but only adds facebook social authentication. Documentation can be found at django-alluth documentation. django-allauth can also be installed by cloning from https://github.com/pennersr/django-allauth
I am running Django 1.9.5 and will install django-allauth 0.25.2 Setup a virtual environment if you haven’t already.

$ pip install django-allauth

Check the packages in your virtual environment with,

$ pip list

You can install the packages in requirements.txt to your virtual environment or whatever environment you are in with,

$ pip install -r requirements.txt

Put the packages in your virtual environment into your requirements.txt file with,

$ pip freeze -> requirements.txt

For this example the django project name is `crudProject’ and the app name is `crudapp’. In settings.py add appropriately to TEMPLATES so that it looks like this,

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [os.path.join(BASE_DIR, "templates")],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
                'django.template.context_processors.request',
            ],
        },
    },
]

Bear in mind,

'django.template.context_processors.request',

may already be in the `context_processors’
Then add to settings.py,

AUTHENTICATION_BACKENDS = (
    # Needed to login by username in Django admin, regardless of `allauth`
    'django.contrib.auth.backends.ModelBackend',

    # `allauth` specific authentication methods, such as login by e-mail
    'allauth.account.auth_backends.AuthenticationBackend',
)

Then add to INSTALLED_APPS

INSTALLED_APPS = (
    ...
    # The Django sites framework is required
    'django.contrib.sites',
    'allauth',
    'allauth.account',
    'allauth.socialaccount',
    'allauth.socialaccount.providers.facebook',
)

and also add to settings.py,

SITE_ID = 1

To urlpatterns = […] in urls.py add,

url(r'^accounts/', include('allauth.urls')),

To see the accounts urls available to you go to localhost:8000/accounts/ be sure not to omit the trailing ‘/’.
Next run a migrate,

$ python manage.py migrate
python manag.py runserver

And go to localhost:8000/accounts/admin You should see in admin, ACCOUNTS, SITES and SOCIALACCOUNTS.
Selection_015

 

We need to go to the Facebook Developer Site. Signup to get account or login with your Facebook credentials if you already have a Facebook account.
In the `My Apps’ pull down menu select `Add a New App’
Selection_016

 

and then click on `Website`
Selection_017

Put in the name of your app which in my case is `crudapp`

 

Selection_018

then continue to `Create New Facebook App ID`. You’ll be asked for an email after that, choose a category for your app. Choose `Education’. Next click `Create App ID`

 

Selection_019

Skipped down to the bottom of the page

 

Selection_021
and for `Site URL:’ Put in localhost:8000, then click `Next’.
Then choose `Login’

 

Selection_022

 

Then go to the `Apps’ pulldown menu at the top right of the page and choose `crudapp`. This should take you to the app’s dashboard and you should see the app ID and secret key which you can reveal my clicking on the `Show’ button.

 

Selection_023

Now go back to your browser and navigate to localhost:8000/admin and go into `SITES` and click on `example.com`

 

Selection_024

 

Change the `Domain name` and `Display name` to the following,

Selection_025
Now localhost:800 has site ID: 1. Back in the admin home click on `Social applications’ under `SOCIALACCOUNTS’.

 

Selection_026
Next click `ADD SOCIAL APPLICATION` which is over on the top right of the screen. Fill in the `Provider`, `Name` and the `Client id` and `Secret key` which we looked at on the facebook developer site. Click the `localhost:8000` under `Available sites`.

 

Selection_027
Move `localhost:8000` across from `Available sites` to `Chosen sites` by clicking the little right arrow between them, while it is highlighted.

 

Selection_029
Then click `SAVE` at the bottom right of the page. Now logout of `admin and navigate to http://localhost:8000/accounts/login`
This should bring up a simple unstyled form,

 

Selection_030
You need to return to the Facebook developer site and under `App Review` set to `Yes` under `Make crudapp public?`

 

Selection_031
Logout of the Facebook developer site and all Facebook accounts and go back to http://localhost:8000/accounts/login/
This will bring you to http://localhost:8000/accounts/loggedin/#_=_ where you will see `Page not found (404)`. To correct this,
In settings.py add,

LOGIN_REDIRECT_URL = '/'

to redirect to the homepage after login.
Next add to your app the a `templates` directory and inside that a directory called `account`. In your directory where you setup your virtual environment you will find a lib directory from there go to,

lib/python2.7/site-packages/allauth/templates

where you should find a base.html templates file copy this to your app/templates directory.

$ cp base.html ~/djangoForum/djangoForum/crudapp/templates/

Then change directory into `account` and you will see html template files for login.html, logout.html, signup.html etc. These files need to be copied to your app’s templates/account directory.

$ cp -r * ~/djangoForum/djangoForum/crudapp/templates/account/

You can now style these templates.

Access the Python shell

Access the Python shell with,

$ python manage.py shell

This will take you to a Python shell which should look like this,
Selection_032
However,

$ python manage.py dbshell

will take you to a ‘sqlite’ shell if it is installed, which looks like this,
Selection_033
This can also be accessed outside of Django with,

$ sqlite3

Selection_034
So continuing now in the Python shell (not dbshell) connect with the models in your app using,

>>> from <appname>.models import <model1>, <model2>

Have a look at everything in the models1

models.objects.all()

Part 7: Django Login, Logout & Registration

Login form:
Adapted from the login tutorial at techand startup.
Create a new app called ‘accounts’

$ python manage.py startapp accounts

Add ‘accounts’ to settings.py

INSTALLED_APPS = [
'accounts',
'django.contrib.admin',
'django.contrib.auth',
.............

Go to crudProject/views.py and add to the top of the file,

from django.shortcuts import render_to_response

Below that add,

def loggedin(request):
return render_to_response('registration/loggedin.html')

If you want to show their username when they login then call their username in the view. Change the loggedin function to:

def loggedin(request):
return render_to_response('registration/loggedin.html',
{'username': request.user.username})

Add the following urls to crudProject/urls.py

 # Auth-related URLs:
url(r'^accounts/login/$', 'django.contrib.auth.views.login', name='login'),
url(r'^accounts/logout/$', 'django.contrib.auth.views.logout', name='logout'),
url(r'^accounts/loggedin/$', 'crudProject.views.loggedin', name='loggedin'),

To the bottom of crudProject/settings.py add,

LOGIN_REDIRECT_URL = '/accounts/loggedin/'

Add login and logout buttons to index.html

<a href="/accounts/logout">logout</a>
<a href="/accounts/login">login</a>

Create a ‘registration’ directory in templates and add ‘login.html’

{% extends "index.html" %}
{% block title %}Log In{% endblock %}
{% block content %}

<form method="post" action="{% url 'django.contrib.auth.views.login' %}">
{% csrf_token %}
<table>
{{ form.as_table }}
</table>

<input type="submit" value="login" />
</form>

{% endblock %}

In templates/registration/loggedin.html put,

{% extends "index.html" %}
{% block title %}Logged In{% endblock %}
{% block content %}

<h1>Welcome {{username}}</h1>
<p>Thank you for logging in.</p>
<p><a href="/accounts/logout/">Logout</a></p>

{% endblock %}

The create templates/registration/logged_out.html

{% extends "index.html" %}
{% block title %}Logged Out{% endblock %}
{% block content %}

<h2>Logged out!</h2>
<p><a href="/accounts/login/">Log back in</a></p>

{% endblock %}

User Registration:
Add to crudProject/views.py

from django.shortcuts import render_to_response
from django.http import HttpResponseRedirect
from django.contrib.auth.forms import UserCreationForm
from django.core.context_processors import csrf

def register(request):
if request.method == 'POST':
form = UserCreationForm(request.POST)
if form.is_valid():
form.save()
return HttpResponseRedirect('/accounts/register/complete')

else:
form = UserCreationForm()
token = {}
token.update(csrf(request))
token['form'] = form

return render_to_response('registration/registration_form.html', token)

def registration_complete(request):
return render_to_response('registration/registration_complete.html')

Add to crudProject/urls.py

# Registration URLs
url(r'^accounts/register/$', 'crudProject.views.register', name='register'),
url(r'^accounts/register/complete/$', 'crudProject.views.registration_complete', name='registration_complete'),
...........................

Part 6: Django CRUD, Update

Continued from Part 5: Django CRUD, Delete

Full code on https://github.com/shanegibney/djangoCRUD

We may wish to update the model ‘Members’. Let us start by adding an edit icon to the table in ‘index.html’.

<td><a href="{% url 'edit_new' item.id %}"><span class="glyphicon glyphicon-pencil"></span></a></td>

If you are not using Bootstrap replace the span tags with ‘edit’.

<td><a href="{% url 'edit_new' item.id %}">edit</a></td>

This will bring us to a url named ‘edit_new’ and also will pass the ‘id’. Create this url in urls.py,

url(r'^edit/(?P\d+)/$', home.edit_new, name='edit_new'),

Django then brings us to the view ‘edit_new’ which we will now create in ‘views.py’,

def edit_new(request, id):
    post = get_object_or_404(Members, pk=id)
    if request.method == "POST":
        form = MemberForm(request.POST, instance=post)
        if form.is_valid():
            post = form.save(commit=False) 
            post.save()
            return redirect('init')
    else:
        form = MemberForm(instance=post)
    return render(request, 'member_edit.html', {'form': form})

This is almost identical to the view ‘member_new’ which we used to create a new instance of the model ‘Members’. But this time we use ‘id’ and ‘pk=id’. Again we the ‘.save()’ method to update the model. It is important to use ‘instance=post’ so that the ‘else’ part of the if statement return the current instance of the model ‘Members’ to ‘edit.html’, otherwise we will have a blank form and nothing to edit.
Django then passes the dictionary {‘form’:form} to ‘member_edit.html’ and everything after that works the same as it did when we used the from to create an instance of the model ‘Members’.
This completes the ‘U’ for ‘update’ in CRUD.

Part 5: Django CRUD, Delete

Continued from Part 4: Django CRUD, Create

Full code on https://github.com/shanegibney/djangoCRUD

Next we will delete an instance, or row from the model ‘Members’. First we will place an icon at the end of the row in the table, which we want to delete. Add the following data cell to the table in ‘index.html’,

<td><a href="{% url 'delete_new' item.id %}"><span class="glyphicon glyphicon-trash"></span></a></td>

If you are not using Bootstrap simply replace the span tags with ‘Delete’.

<td><a href="{% url 'delete_new' item.id %}">Delete</a></td>

We can see here that the hyper-link takes us to a url called ‘delete_new’. Also the ‘id’ of the row is passed. We need to create this url in ‘urls.py’ add the following,

url(r'^delete/(?P\d+)/$', home.delete_new, name='delete_new'),

This creates a url for example ‘delete/27’ where 27 is the ‘id’ and this is then passed to the view ‘delete_new’.
Let’s create this view in ‘views.py’. We will need to import ‘get_object_or_404’.

from django.shortcuts import render, get_object_or_404
........
 
def delete_new(request, id):
     post = get_object_or_404(Members, pk=id)
     post.delete()
     return redirect(init)

Here we see that we have passed ‘id’ along with ‘request’. ‘post’ is passed the instance of the ‘Members’ model with pk=id. This object ‘post’ has a method ‘.delete()’, which in the case deletes an instance of the ‘Members’ model. An instance of a class, i.e. ‘Members’ is a row in the database. Next we use ‘redirect’ to go back to the ‘init’ view which renders ‘index.html’.

Now we have an example of the ‘D’ for ‘delete’ in CRUD.

Part 6: Django CRUD, Update

Part 4: Django CRUD, Create

Continued from Part 3: Django CRUD, Display Models

Full code on https://github.com/shanegibney/djangoCRUD

Before starting this section it would be a good idea to include Bootstrap to make use of the classes in the html, but it is not necessary. It is only to make it look nice! To do that add the following lines inside the head tags of index.html.

<link rel="stylesheet" href="//maxcdn.bootstrapcdn.com/bootstrap/3.2.0/css/bootstrap.min.css">
    <link rel="stylesheet" href="//maxcdn.bootstrapcdn.com/bootstrap/3.2.0/css/bootstrap-theme.min.css">

CRUD is an acronym for Create, Read, Update and Delete. First we would like to make a form and send this data to the database or our ‘Members’ model. This will be the Create part of CRUD.
First we need to create a form. But to do that we need to tell Django which fields from the model members to put in the form. Create a new file called ‘forms.py’ and place it in the same place as ‘models.py’.

‘forms.py’ must import forms from Django and your model Members from models. This done by adding to the top of ‘forms.py’

from django import forms
from .models import Members

We create a new model here for the form, but it takes from the current model ‘Members’. This is done with a ‘class Meta’ and we can choose which fields from ‘Members’ we want in the form,

 class MemberForm(forms.ModelForm):
    class Meta:
        model = Members
        # which fields to put in form
        fields = ('first_name', 'last_name', 'description')

We need a link to the url which we are calling ‘member_new’. So in urls.py add in,

url(r'^post/new/$', home.member_new, name='member_new'),

This gives the url a name of ‘member_new’. To get to this url you need a hyper-link in ‘index.html’,

<a href="{% url 'member_new' %}">Add a post</a>

This causes Django to look in urls.py for a url named ‘member_new’ where it is associated with the url ‘post/new/’. Note that is not actually a path called ‘post/new/’. Then Django looks through all these urls again to find the url ‘post/new/’ and see what view it is associated with. In this case the view is ‘home.members_new’. Now will create this view in ‘views.py’,

def member_new(request):

Django first checks if the “POST” method exists. Of course it doesn’t because we haven’t used a form yet. Therefore this section is ignored and Django carries on to the ‘else’.

    if request.method == "POST":
        ..... we will come here after the form is sent ......
    else:
        form = MemberForm()
    return render(request, 'member_edit.html', {'form': form})

The ‘MemberForm’ which we created earlier and has the fields in it is now passed to ‘form’, and the last line renders ‘member_edit.html’ by sending it {‘form’: form}.
We need to create ‘member_edit.html’ in the templates folder. This file will extend ‘index.html’. To create the form itself we only need to wrap in form tags {{ form.as_p }} like this,

{% extends 'index.html' %}

{% block content %}
    <th1>Member form&lt/h1>
    <form method="POST" class="post-form">{% csrf_token %}
        {{ form.as_p }}in the form
        <button type="submit" class="save btn btn-default">Save<button>
    <form>
{% endblock %}

This creates a block called ‘content’ which will be placed by Django in ‘index.html’ if we add to ‘index.html’ the following,

{% block content %}
{% endblock %}

When you run the server you should have a link on the ‘index.html’ page to a form with fields. But you won’t be able to submit because we haven’t finished writing the view ‘member_new’, which is where this form will submit to.

But this time “POST” does exist and so the if statement is not ignored. We need to fill this view in so that completed it looks like this,

def member_new(request):
    if request.method == "POST":
        form = MemberForm(request.POST)
        if form.is_valid():
            post = form.save(commit=False)
            post.save()
            return redirect('post_detail', pk=post.pk)
    else:
        form = MemberForm()
    return render(request, 'member_edit.html', {'form': form})

Note that we are using ‘redirect’, this needs to be imported by placing the following line at the top of ‘views.py’,

from django.shortcuts import redirect

You can see this passes everything in “POST” to form and validates ‘form’ with ‘.is_valid’. The line ‘form.save…’ saves the form data to memory, not to the database yet. ‘post.save()’ does that. Therefore this is where a new row is added to the database or where the actual ‘create’ in CRUD happens. The next line redirects us to the ‘init’ view. Which recreates the home page and the table of data from the ‘Members’ model.
This concludes the ‘C’ or ‘create’ in CRUD and we have already dealt with the ‘R’ for ‘read’.

Part 5: Django CRUD, Delete

Part 3: Django CRUD, Display Models

Continued from Part 2: Setting up Django, URL’s & Views

Full code on https://github.com/shanegibney/djangoCRUD

Add to the crudapp/models.py file

from django.db import models

In the same file create a model called ‘Members’ with columns for ‘first_name’, ‘last_name’ and ‘description’. This is equivalent to a table called ‘Members’ with fields for ‘first_name’, ‘last_name’ and ‘description’.

class Members(models.Model):
    first_name = models.CharField(max_length=200)
    last_name = models.CharField(max_length=200)
    description = models.CharField(max_length=600)
    # pub_date = models.DateTimeField('date published')
    def __str__(self):
        return ' '. join([ self.first_name, self.last_name, ])

Give the model an admin interface, in crudapp/admin.py

from .models import Members

admin.site.register(Members)

Go to http://localhost:8000/admin and you will see a model called ‘Memberss’ with effectively rows of the database or as we say in Django instances of the class Member. Note Django places an extra ‘s’ on the class name in admin. Using the Django admin you can now view, edit and delete this model.

Next we will change the cruduapp/views.py file so that it takes data from the model or class Members

from django.shortcuts import render
from .models import Members
# Create your views here.

def init(request):
    details = Members.objects.all()
    context = {'details': details}
    return render(request, 'index.html', context)

This passes all the data in the class Members to index.html where we can display it,

{% load staticfiles %}
<!DOCTYPE html>
<html lang="en">
<head>
</head>
<body>
  {% if details %}
  <ul>
    {% for item in details %}
    <li>{{ item.description}}</li>
    {% endfor %}
  </ul>
  {% else %}
  <p>No polls are available.</p>
  {% endif %}
</body>

Part 4: Django CRUD. Create

Part 2: Setting up Django, URLs & Views

Continued from Part 1: Setting up Django and virtualenv
Will will add a url to out crudapp. Add the following into urls.py

...
from crudapp import views as home

urlpatterns = [
    ...
    url(r'^$', home.init),
]

This means when we go to the root url http://localhost:8000 we will be directed to views where Django will look for a function or view called init(). Let us create that next. Add the following to crudapp/views.py

def init(request):
    return render(request, 'index.html')

This will direct us to index.html. However, index.html needs to be created in a new folder called templates. Inside the root folder djangoCRUD create templates/index.html

<!DOCTYPE html>
<html lang="en">
<head>

</head>
<body>
  <p>hello forum</p>
</body>

We have not told Django where the templates folder is. Add

os.path.join(BASE_DIR, "templates")

to settings.py like this,

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [os.path.join(BASE_DIR, "templates")],
....

Part 3: Django CRUD, Display Models

Accessing an sqlite3 database

To access sqlite3 from the command line.

$ sqlite3

Now you are in the sqlite3 shell. Next connect to an sqlite3 database called db.sqlite3

sqlite> .open db.sqlite3

List tables in database

sqlite> .tables

Let us see what is in the table called table_name

sqlite> select * from table_name;

Not forgetting the trailing semicolon.
Ctrl+D will allow you exit this shell.
To view a table’s field names,

sqlite> PRAGMA table_info(table_name);

To view a table’s schema,

sqlite> .schema table_name

Setting up Python Virtual Environment using Command Line

It is easy to set up virtual environments using an IDE such as PyCharm, but here is how to do it with the command line. Here the virtual environment is called ‘gummienv’.
First set up a directory and cd into it.

$ mkdir gummi
$ cd gummi/
$ sudo pip install virtualenv
$ virtualenv gummienv,

Activate the virtual environment using,

$ source gummienv/bin/activate

Notice the name of the virtual environment is now in the command line prompt.

Selection_010
The virtual environment can be deactivated with,

$ deactivate

Python Dictionary into HTML Template

The following Python code will find instances of {body} in a file template.html and replace it with a string. Then a new file index.html is created with this change.

generate.py

with open('template.html','r') as template_file:
    template = template_file.read()

tokens = {'title': 'My Title in a Dictionary', 'header':'Stuff for heading', 'body': 'The body stuff', 'footer': 'This is the footer'}
content = template

for token in tokens:
    content = content.replace('{' + token + '}', tokens[token])

with open('index.html', 'w') as content_file:
     content_file.write(content)

index.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>{title}</title>
</head>
    <div><p>{header}</p></div>
    <div><p>{body}</p></div>
    <div><p>{footer}</p></div>
</html>

Simple Python into HTML

The following Python code will find instances of {body} in a file template.html and replace it with a string. Then a new file index.html is created with this change.

generate.py

with open('template.html','r') as template_file:
    template = template_file.read()

content = template
tokens = {'body': 'Body stuff can go here'}
replaced = 'Body stuff can go here.'
content = content.replace('{' + 'body' + '}', replaced)

with open('index.html', 'w') as content_file:
     content_file.write(content)

index.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>Title</title>
</head>
    <div><p>{body}</p></div>
</html>

Running Python unittests from Command Line

This script will allow a Python unittest to be run from the command line.

import unittest
import testFile
suite = unittest.TestLoader().loadTestsFromModule(testFile)
unittest.TextTestRunner().run(suite)

testFile.py

import unittest
from fun import add

class test_function(unittest.TestCase):
     def test_one(self):
           result = add(2,3)
           self.assertEquals(result, 5)

fun.py

def add(a,b):
     return a+b

Using Python to Connect to a MongoDB

This code uses Python to connect to a mongoDB, allowing for print out of the names of all the databases and the names of the collections in a given database, which is in this example is called ‘mydb’.

import pymongo
def mongo_connect():
    try:
         conn = pymongo.MongoClient()
         print "Mongo is connected"
         return conn
    except pymongo.errors.ConnectionFailure, e:
         print "Could not connect to MongoDB: %s" % e
conn  = mongo_connect()
print "Database names: %s" % conn.database_names()
db = conn.mydb
coll = db.collection_names()

print "Database name: %s" % db.name
print "Collection names: %s" % coll

Simple Python Unit Test

This simple Python unit-test tests two function. The first function adds two numbers and the second function uses list comprehensions to make a list of numbers from to 0 to n which are divisible by m.
testFile.py

import unittest
from add import addition, make_list

class test_add_function(unittest.TestCase):

    def test_for_add(self):
        result = addition(1, 5)
        self.assertEqual(result, 6)

    def test_multiples_of_3_less_than_10(self):
        result = make_list(10, 3)
        self.assertEquals(result, [0, 3, 6, 9])

The file containing the functions,
add.py

def addition(a,b):
    return a+b

def make_list(n, m):
    return [i for i in range(n) if i%m == 0]

Part 1: Setting up Django and virtualenv

First create a new directory for your project

$ mkdir djangoCRUD

Move to that directory

$ cd djangoCRUD

Set up a virtual environment.

$ virtualenv .

$ source bin/activate

Use pip list to see the modules in this virtual environment.

$ pip list

Now we can install the Django package

$ pip install django

Run pip list again and you will see that django is now part of this environment.

If requested to do so, it is a good idea to upgrade pip,

$ pip install --upgrade pip

Next set up your Django project

$ django-admin startproject crudProject

Change directory into your Django project directory,

$ cd crudProject

Now create an app for your project. To do this you must be in the same directory as the manage.py file.

$ python manage.py startapp crudapp

Note: it is also possible to start a new app using with,

$ django-admin startapp crudapp

Make the database tables with,

$ python manage.py makemigrations
$ python manage.py migrate

After this has been done we can create a super user.

$ python manage.py createsuperuser

Next we will turn on the server,

$ python manage.py runserver

Navigate in your browser to localhost:8000

sqlite is the default database. Ignore the following if you do not want to use a mysql database.
Presuming you have a mysql database set up and running with a user associated with it then in mysite/settings.py

DATABASES = {
    `default' = {
        `ENGINE' : `django.db.backends.mysql',
        `NAME' : `&lt db-name>',
        `USER' : `&lt db-user>',
        `PASSWORD' : `&lt password>',
        `HOST' : `localhost', 
        `PORT' : `',
   }
}

Part 2: Setting up Django, URLs & Views