Python code creates a database from .csv

Content

  1. Documentation
  2. Code

1. Documentation

csv2sqliteDB.py:
1. Creates an sqlite Database (from SQLITE_FILEPATH) then
2. Creates and Loads TABLE_NAME (adding id INTEGER PRIMARY KEY )
FROM CSV_FILEPATH (with header, NO id field necessary)
INTO DATABASE_FILEPATH
CREATING DATABASE_FILEPATH if it doesn’t exist.

Usage:   csv2sqliteDB.py TABLE_NAME CSV_FILEPATH  SQLITE_FILEPATH
Example: csv2sqliteDB.py contacts   contacts2.csv contactsDB2id.db

2. Code


#!/usr/bin/python
#! python3
#! python2
# -*- coding: utf-8 -*-
"""csv2sqliteDB.py Creates and Loads TABLE_NAME (adding id INTEGER PRIMARY KEY )
      from CSV_FILEPATH (with header, NO id field necessary)
      into DATABASE_FILEPATH
          creating DATABASE_FILEPATH if it doesn't exist

Usage:   csv2sqliteDB.py TABLE_NAME CSV_FILEPATH  SQLITE_FILEPATH
Example: csv2sqliteDB.py contacts   contacts2.csv contactsDB2id.db
"""

# see:
    # Importing a CSV file into a sqlite3 - http://stackoverflow.com/a/2888042/601770
    # csv file using python with headers intact - http://stackoverflow.com/a/3428633/601770

import os, sys, csv, sqlite3

def csv2sqliteDB(table_name, csv_filepath, database_filepath):
    conn, cursor = createDatabaseIfNotExist(database_filepath)
    nlines = sum(1 for line in open(csv_filepath)) # http://stackoverflow.com/a/36973958/601770
    with open(csv_filepath, 'r') as f:
        reader = csv.reader(f)
        headerL = reader.__next__()

        createTable(table_name, conn, cursor, headerL)

        loadTable(table_name, conn, cursor, headerL, reader, nlines)
    conn.close()

def createDatabaseIfNotExist(database_filepath):
    # if we error, we rollback automatically, else commit!
    with sqlite3.connect(database_filepath) as conn:
        cursor = conn.cursor()
        cursor.execute('SELECT SQLITE_VERSION()')
        data = cursor.fetchone()
        print('SQLite version:', data)
        return conn, cursor

def createTable(table_name, conn, cursor,  headerL):
    """
    :param table_name:
    :param conn:
    :param cursor:
    :param header:
    :return:
    """
    # http://stackoverflow.com/a/12432311/601770
    # http: // stackoverflow.com / a / 19730169 / 601770

    textL = len(headerL)*'TEXT'.split()
    def_str = ', '.join( ['id INTEGER PRIMARY KEY'] + [i[0]+' '+i[1] for i in list(zip(headerL, textL))] )
    script_str = """DROP TABLE IF EXISTS %s;
        CREATE TABLE %s (%s);
    """%(table_name, table_name, def_str) # checks to see if table exists and makes a fresh table.

    cursor.executescript(script_str)


def loadTable(table_name, conn, cursor, headerL, reader, nlines):
    insert_str_model = "INSERT INTO {table_name} ({col_names}) VALUES({q_marks});"
    col_names = ','.join(headerL)
    q_marks = ','.join(len(headerL)*'?'.split())

    insert_str = insert_str_model.format(table_name=table_name, col_names=col_names, q_marks=q_marks)

    # DO THE INSERTS
    line_count = 0
    for row in reader:
        line_count += 1
        if line_count%10 == 1: print('line %s of %s'%(line_count, nlines))
        cursor.execute(insert_str, row)
        conn.commit()


NUM_ARGS = 3
def main():
    args = sys.argv[1:]
    if len(args) != NUM_ARGS or "-h" in args or "--help" in args:
        print (__doc__)
        sys.exit(2)
    csv2sqliteDB(args[0], args[1], args[2])

if __name__ == '__main__':
    main()

Teach Kids Coding – Turtle, Python, IDLE

Advantage – Immediate Feedback

As the student repeatedly presses the Debug Control’s “Over” button, she will be able to see the Turtle execute the commands she has given it.

Content

  • Summary
  • Video Link
  • Technique Steps
  • Code

Summary

This post will demonstrate a technique for teaching kids (or anyone) coding, using Turtle Graphics, Python, Python IDLE, Python IDLE’s Debug Control and Statement Execution Stepping.

This technique should work for Windows, Mac, and Linux (e.g. Raspberry Pi).

Video Link

Here is a link to a YouTube Video for this post.

Technique Steps

  1. Open your File Manager
    • “File Explorer” in Windows
    • “Finder” in Mac
    • “File Manager” in Raspberry Pi
  2. Navigate to the folder [CodeFolderPath] in which you have your code to be used. Most File Managers let you copy the path to a command line.
  3. Open a Command Line Window
    • “Command Prompt” in Windows
    • “Terminal” in Mac and Raspberry Pi
  4. At the Command Line, change directory to [CodeFolderPath], whatever that is.
    • example:
    • $ cd ~/pythonPjs/MasonPjs
  5. At the Command Line, run IDLE.
    • $ idle
    • This will bring up the IDLE Shell Window.
  6. In the IDLE Shell Window’s menu bar, click File Open and open the appropriate code file.
    • This will bring up the IDLE Code Window.
  7. In the IDLE Shell Window’s menu bar, click Debug/Debugger.
    • This will bring up the IDLE Debug Control Window.
  8. In the Debug Control Window:
    • UnTick the Stack Checkbox.
    • Tick the Source Checkbox.
  9. Minimize the IDLE Shell, File Manager and Command Line Windows. We want the student to focus on the important windows.
    • IDLE Debug Control
    • IDLE Code
    • Turtle Graphics
  10. Position the Debug Control & IDLE Code windows so you’ll be able to see them AND the Turtle Graphics Window SIMULTANEOUSLY. I plan on Turtle Graphics Window to be on the RIGHT of the screen. So I put:
    • Debug Contol – Top Left of the screen.
    • IDLE Code – Bottom Left of the screen. I make it about the same WIDTH as the Debug Control Window.
  11. In the IDLE Code window click on File/Run/Run Module
    • RESIZE it to be above the IDLE Code Window.
    • ALSO Minimize the IDLE Shell Window that (annoyingly🙂 ) popped up again.
  12. Click on Debug Control Window’s “Over” button until the Turtle Graphics window appears. NOTICE with each click of “Over”.
    • NOTICE with each click, A NEW LINE in the Code Window is highlighted.
    • NOTICE when forward() or left() commands are executed, the Turtle Graphics Window responds with the appropriate action.
    • NOTICE after the line A1pos = pos(); A2abs_pos = abs(pos()) has been executed, the Debug Control Window contains 2 lines under the “Locals”
    • A1pos – (200.00,0.00) or something like this value
    • A2abs_pos – 200.00 or something like this value
      • Keeping track of the current values of these items.
    • NOTICE that after we execute the if abs(pos()) < 1: statement, we loop back to just under the while True: statement.
  13. Click on "Over" for about 30 more times. NOTICE the Turtle Graphics window cooking along, OBEYING the commands in the Code Window.🙂
  14. Right Click on the break Line & select "Set Breakpoint".
    • NOTICE the yellow background.
  15. In the Debug Control Window Click on the "Go" button. This will run until just before we execute the breakpoint line.
    • WATCH the Trutle Graphics window do its thing.
  16. Click "Over".
    • NOTICE we exit the while True: loop.
  17. Click "Over".
    • NOTICE the end_fill() statement colors the star yellow.
  18. Click "Over" one last time.
    • NOTICE we are DONE.🙂

Code

Here is the code we used in the video.

from turtle import *
setup (width=500, height=500, startx=500)
color('red', 'yellow')
begin_fill()
while True:
    forward(200)
    left(170)
    A1pos = pos(); A2abs_pos = abs(pos()) 
    if abs(pos()) < 1:
        break
end_fill()
done()

Naive Spot-Check of AI Algorithms

More Adventures in AI

A “Back to Basics” RESET

I found the work of Jason Brownlee to help me get “Back to Basics” with AI.

Currently i am following his EXCELLENT Python Machine Learning Mini-Course.

At Lesson 9, He suggests one should “Spot-Check Algorithms”.

I took a naive approach employing the following “Analysis Code” section below to generate the resulting “Analysis Grid” section below.

I am hoping to use this post to get information in the Comments Section about how to use Data Preparation for various (Dataset, Model(Algorithm), Scoring) combinations, AND which (Dataset, Model(Algorithm), Scoring) combinations are JUST INCOMPATIBLE.

Methodology

To generate the Algorithm Spot-Check Cases i cross multiplied:

  1. (3) Datasets
    1.1 Boston House Price Data
    1.2 Iris Data
    1.3 Pima Indians Diabetes Data
  2. (4) Models(Algorithms)
    2.1 KNeighborsRegressor
    2.2 LinearRegression
    2.3 LogisticRegression
    2.4 LinearDiscriminantAnalysis
  3. (3) Scorings
    3.1 accuracy
    3.2 neg_mean_squared_error
    3.3 neg_log_loss

for a total of 36 Spot-Check Cases.

I then ran each case against a
kfold = sklearn.model_selection.KFold(n_splits=10, random_state=7)
with
results = sklearn.model_selection.cross_val_score(model, X, Y, cv=kfold, scoring=scoring)

Summary Analysis

There were 11 out of 36 Cases that returned numerical results. The other 25 Cases returned Errors or Warnings. The full Analysis can be seen in the “Analysis Grid” section below.

Question Reiteration, Answer, Further Study

Question Reiteration

How can I use Data Preparation for various (Dataset, Model(Algorithm), Scoring) combinations, AND which (Dataset, Model(Algorithm), Scoring) combinations are JUST INCOMPATIBLE.

I posted this question to Jason Brownlee in the Comments on Python Machine Learning Mini-Course. The Answer is his response to me there.

Answer

From Jason Brownlee in the Comments on Python Machine Learning Mini-Course:

Nice post and great question Joe.

Spot checking is to discover which algorithms look good on one given dataset. Not across datasets.

You may need to group algorithms by their expectations then prepare data for each group.

Most machine learning algorithms expect data to have numeric input values and an integer encoded or one hot encoded output value for classification. This is a good normalized view of a dataset to construct.

Here’s a tutorial that shows how to spot check 7 machine learning algorithms on one problem in Python, Spot-Check Regression Machine Learning Algorithms in Python with scikit-learn.

Further Study

From Jason’s Answer, above, I need to study:

  • Algorithm Groups
  • Algorithm Group Expectations
  • Data Preparation

Analysis Code

#! python3
# -*- coding: utf-8 -*-
"""analyzeAlgorithmsMod.py module contains  AnalyzeAlgorithms class.
    It aids in the comparison of Machine Learning Algorithms(Models)
       for a particular dataset.
"""
import os, sys, shutil, time, datetime, urllib, tarfile, zipfile, csv, io, copy, itertools, six

import numpy as np
import pandas as pd
import matplotlib
import matplotlib.pyplot as plt
import scipy.stats
import sklearn
import sklearn.preprocessing
import sklearn.linear_model

import sklearn.cross_validation
# import sklearn.model_selection

import sklearn.ensemble
import sklearn.metrics
import sklearn.discriminant_analysis
import sklearn.preprocessing
import itertools
import textwrap
from pprint import pprint as pp

import warnings
warnings.filterwarnings("error")


def reportInModule(model, scoring, results):
    print("\n\n{model: '%s', scoring: '%s'}:\n    results summary: %.3f mean (%.3f) std" % (type(model).__name__, scoring, results.mean(), results.std(),))
    print("    sorted(results):")
    pp(sorted(results), indent=8)


class AnalyzeAlgorithms:
    """Aids in the comparison of Machine Learning Algorithms
       for a particular dataset.
           
Python Machine Learning Mini-Course
""" @staticmethod def calcX_Y(dataFrame): array = dataFrame.values X = array[:, 0:dataFrame.shape[1]-1] Y = array[:, dataFrame.shape[1]-1] return X, Y def __init__(self, datasetInfoTupleList, modelList, scoringStrList, kfold): self.datasetInfoTupleList = datasetInfoTupleList # (datasetTitle, csvFilePath, delim_whitespace, columnNamesList) self.modelList = modelList self.scoringStrList = scoringStrList self.kfold = kfold def analyzeAlgorithms(self): for datasetInfoTuple in self.datasetInfoTupleList: # (datasetTitle, csvFilePath, delim_whitespace, columnNamesList) self.datasetTitle, self.csvFilePath, self.delim_whitespace, self.columnNamesList = datasetInfoTuple self.df = pd.read_csv(self.csvFilePath, delim_whitespace=self.delim_whitespace, names=self.columnNamesList) self.X, self.Y = AnalyzeAlgorithms.calcX_Y(self.df) for model in self.modelList: for scoring in self.scoringStrList: print("trying|%s|%s|%s|"%(self.datasetTitle, type(model).__name__, scoring),end='') results = self.genResults(model, scoring, self.X, self.Y, self.kfold) self.report_short(datasetInfoTuple, model, scoring, results) def genResults(self, model, scoring, X, Y, kfold): try: results = sklearn.model_selection.cross_val_score(model, X, Y, cv=kfold, scoring=scoring) except: results = "Error: %s"%( sys.exc_info()[1] ) return results def report_short(self, datasetInfoTuple, model, scoring, results): datasetTitle = datasetInfoTuple[0] if isinstance(results, six.string_types): results_short = results.splitlines()[0] print(results_short) else: results_short = results.mean() print(results_short) def report(self, model, scoring, results): if isinstance(results, six.string_types): # Error print("\n\n{model: '%s', scoring: '%s'}:\n results ERROR: %s" % (type(model).__name__, scoring, results) ) else: print("\n\n{model: '%s', scoring: '%s'}:\n results summary: %.3f mean (%.3f) std" % (type(model).__name__, scoring, results.mean(), results.std(),)) print(" sorted(results):") pp(sorted(results), indent=8) def test(): datasetInfoTupleList = [ ('Boston House Price Data', # datasetTitle r'C:\BLA\BLA\BLA\data\BostonHousing\housing.data.txt', # csvFilePath True, # delim_whitespace for csv boolean ['CRIM', 'ZN', 'INDUS', 'CHAS', 'NOX', 'RM', 'AGE', 'DIS', 'RAD', 'TAX', 'PTRATIO', 'B', 'LSTAT', 'MEDV'], # columnNames ), ('Iris Data', # datasetTitle r'C:\BLA\BLA\BLA\data\iris\iris.data.txt', # csvFilePath False, # delim_whitespace for csv boolean ['sepalLength', 'sepalWidth', 'petalLength', 'petalWidth', 'class'], # columnNames ), ('Pima Indians Diabetes Data', # datasetTitle r'C:\BLA\BLA\BLA\data\PimaIndiansDiabetes\pima-indians-diabetes.data.txt', # csvFilePath False, # delim_whitespace for csv boolean ['preg', 'plas', 'pres', 'skin', 'test', 'mass', 'pedi', 'age', 'class'], # columnNames ), ] modelList = [ sklearn.neighbors.KNeighborsRegressor(), sklearn.linear_model.LinearRegression(), sklearn.linear_model.LogisticRegression(), sklearn.discriminant_analysis.LinearDiscriminantAnalysis(), ] scoringStrList = [ 'accuracy', 'neg_mean_squared_error', 'neg_log_loss', ] kfold = sklearn.model_selection.KFold(n_splits=10, random_state=7) analysis02 = AnalyzeAlgorithms(datasetInfoTupleList, modelList, scoringStrList, kfold) analysis02.analyzeAlgorithms() def main(): test() if __name__ == '__main__': main() output = """ C:\Python35\python.exe C:/BLA/BLA/MachineLearningMasteryPj/AnalyzeAlgorithmsPkg/analyzeAlgorithmsMod.py C:\Python35\lib\site-packages\sklearn\cross_validation.py:44: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20. "This module will be removed in 0.20.", DeprecationWarning) trying|Boston House Price Data|KNeighborsRegressor|accuracy|Error: continuous is not supported trying|Boston House Price Data|KNeighborsRegressor|neg_mean_squared_error|-107.28683898 trying|Boston House Price Data|KNeighborsRegressor|neg_log_loss|Error: 'KNeighborsRegressor' object has no attribute 'predict_proba' trying|Boston House Price Data|LinearRegression|accuracy|Error: continuous is not supported trying|Boston House Price Data|LinearRegression|neg_mean_squared_error|-34.7052559445 trying|Boston House Price Data|LinearRegression|neg_log_loss|Error: 'LinearRegression' object has no attribute 'predict_proba' trying|Boston House Price Data|LogisticRegression|accuracy|Error: Unknown label type: 'continuous' trying|Boston House Price Data|LogisticRegression|neg_mean_squared_error|Error: Unknown label type: 'continuous' trying|Boston House Price Data|LogisticRegression|neg_log_loss|Error: Unknown label type: 'continuous' trying|Boston House Price Data|LinearDiscriminantAnalysis|accuracy|Error: Unknown label type: (array([ 20.5, 25. , 23.4, 18.9, 35.4, 24.7, 31.6, 23.3, 19.6, trying|Boston House Price Data|LinearDiscriminantAnalysis|neg_mean_squared_error|Error: Unknown label type: (array([ 20.5, 25. , 23.4, 18.9, 35.4, 24.7, 31.6, 23.3, 19.6, trying|Boston House Price Data|LinearDiscriminantAnalysis|neg_log_loss|Error: Unknown label type: (array([ 20.5, 25. , 23.4, 18.9, 35.4, 24.7, 31.6, 23.3, 19.6, trying|Iris Data|KNeighborsRegressor|accuracy|Error: unsupported operand type(s) for /: 'str' and 'int' trying|Iris Data|KNeighborsRegressor|neg_mean_squared_error|Error: unsupported operand type(s) for /: 'str' and 'int' trying|Iris Data|KNeighborsRegressor|neg_log_loss|Error: 'KNeighborsRegressor' object has no attribute 'predict_proba' trying|Iris Data|LinearRegression|accuracy|Error: could not convert string to float: 'Iris-virginica' trying|Iris Data|LinearRegression|neg_mean_squared_error|Error: could not convert string to float: 'Iris-virginica' trying|Iris Data|LinearRegression|neg_log_loss|Error: could not convert string to float: 'Iris-virginica' trying|Iris Data|LogisticRegression|accuracy|0.88 trying|Iris Data|LogisticRegression|neg_mean_squared_error|Error: could not convert string to float: 'Iris-setosa' trying|Iris Data|LogisticRegression|neg_log_loss|Error: y_true contains only one label (Iris-setosa). Please provide the true labels explicitly through the labels argument. trying|Iris Data|LinearDiscriminantAnalysis|accuracy|Error: The priors do not sum to 1. Renormalizing trying|Iris Data|LinearDiscriminantAnalysis|neg_mean_squared_error|Error: The priors do not sum to 1. Renormalizing trying|Iris Data|LinearDiscriminantAnalysis|neg_log_loss|Error: The priors do not sum to 1. Renormalizing trying|Pima Indians Diabetes Data|KNeighborsRegressor|accuracy|Error: Can't handle mix of binary and continuous trying|Pima Indians Diabetes Data|KNeighborsRegressor|neg_mean_squared_error|-0.196342447027 trying|Pima Indians Diabetes Data|KNeighborsRegressor|neg_log_loss|Error: 'KNeighborsRegressor' object has no attribute 'predict_proba' trying|Pima Indians Diabetes Data|LinearRegression|accuracy|Error: Can't handle mix of binary and continuous trying|Pima Indians Diabetes Data|LinearRegression|neg_mean_squared_error|-0.162812506544 trying|Pima Indians Diabetes Data|LinearRegression|neg_log_loss|Error: 'LinearRegression' object has no attribute 'predict_proba' trying|Pima Indians Diabetes Data|LogisticRegression|accuracy|0.76951469583 trying|Pima Indians Diabetes Data|LogisticRegression|neg_mean_squared_error|-0.23048530417 trying|Pima Indians Diabetes Data|LogisticRegression|neg_log_loss|-0.492545522852 trying|Pima Indians Diabetes Data|LinearDiscriminantAnalysis|accuracy|0.773462064252 trying|Pima Indians Diabetes Data|LinearDiscriminantAnalysis|neg_mean_squared_error|-0.226537935748 trying|Pima Indians Diabetes Data|LinearDiscriminantAnalysis|neg_log_loss|-0.485655330102 Process finished with exit code 0 """

Analysis Grid

Here’s the Analysis Grid copied from an Excel Spreadsheet.


| Dataset                    | Model                      | Scoring                | Result(Error or Mean(result)                                                                                                    | Joe's Comment                                                                     |
|----------------------------|----------------------------|------------------------|---------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------|
| Boston House Price Data    | KNeighborsRegressor        | accuracy               | Error: continuous is not supported                                                                                              | Data Prep? (Boston, [KNeighborsRegressor, LinearRegression], accuracy)            |
| Boston House Price Data    | KNeighborsRegressor        | neg_mean_squared_error | -107.286839                                                                                                                     |                                                                                   |
| Boston House Price Data    | KNeighborsRegressor        | neg_log_loss           | Error: 'KNeighborsRegressor' object has no attribute 'predict_proba'                                                            | NG: (KNeighborsRegressor, neg_log_loss)                                           |
| Boston House Price Data    | LinearRegression           | accuracy               | Error: continuous is not supported                                                                                              | Data Prep? (Boston, [KNeighborsRegressor, LinearRegression], accuracy)            |
| Boston House Price Data    | LinearRegression           | neg_mean_squared_error | -34.70525594                                                                                                                    |                                                                                   |
| Boston House Price Data    | LinearRegression           | neg_log_loss           | Error: 'LinearRegression' object has no attribute 'predict_proba'                                                               | Data Prep? Or Model-Scoring?([Boston, Pima], LinearRegression,   neg_log_loss)    |
| Boston House Price Data    | LogisticRegression         | accuracy               | Error: Unknown label type: 'continuous'                                                                                         | Data Prep (Boston, LogisticRegression, *)                                         |
| Boston House Price Data    | LogisticRegression         | neg_mean_squared_error | Error: Unknown label type: 'continuous'                                                                                         | Data Prep (Boston, LogisticRegression, *)                                         |
| Boston House Price Data    | LogisticRegression         | neg_log_loss           | Error: Unknown label type: 'continuous'                                                                                         | Data Prep (Boston, LogisticRegression, *)                                         |
| Boston House Price Data    | LinearDiscriminantAnalysis | accuracy               | Error: Unknown label type: (array([ 20.5,    25. ,  23.4,  18.9,    35.4,  24.7,  31.6,    23.3,  19.6,                         | Data Prep (Boston, LinearDiscriminantAnalysis *)                                  |
| Boston House Price Data    | LinearDiscriminantAnalysis | neg_mean_squared_error | Error: Unknown label type: (array([ 20.5,    25. ,  23.4,  18.9,    35.4,  24.7,  31.6,    23.3,  19.6,                         | Data Prep (Boston, LinearDiscriminantAnalysis *)                                  |
| Boston House Price Data    | LinearDiscriminantAnalysis | neg_log_loss           | Error: Unknown label type: (array([ 20.5,    25. ,  23.4,  18.9,    35.4,  24.7,  31.6,    23.3,  19.6,                         | Data Prep (Boston, LinearDiscriminantAnalysis *)                                  |
| Iris Data                  | KNeighborsRegressor        | accuracy               | Error: unsupported operand type(s) for /: 'str' and 'int'                                                                       | Data Prep: (Iris Data, KNeighborsRegressor,  [accuracy, neg_mean_squared_error] ) |
| Iris Data                  | KNeighborsRegressor        | neg_mean_squared_error | Error: unsupported operand type(s) for /: 'str' and 'int'                                                                       | Data Prep: (Iris Data, KNeighborsRegressor,  [accuracy, neg_mean_squared_error] ) |
| Iris Data                  | KNeighborsRegressor        | neg_log_loss           | Error: 'KNeighborsRegressor' object has no attribute 'predict_proba'                                                            | NG: (KNeighborsRegressor, neg_log_loss)                                           |
| Iris Data                  | LinearRegression           | accuracy               | Error: could not convert string to float: 'Iris-virginica'                                                                      | Data Prep: (Iris Data, LinearRegression, *)                                       |
| Iris Data                  | LinearRegression           | neg_mean_squared_error | Error: could not convert string to float: 'Iris-virginica'                                                                      | Data Prep: (Iris Data, LinearRegression, *)                                       |
| Iris Data                  | LinearRegression           | neg_log_loss           | Error: could not convert string to float: 'Iris-virginica'                                                                      | Data Prep: (Iris Data, LinearRegression, *)                                       |
| Iris Data                  | LogisticRegression         | accuracy               | 0.88                                                                                                                            |                                                                                   |
| Iris Data                  | LogisticRegression         | neg_mean_squared_error | Error: could not convert string to float: 'Iris-setosa'                                                                         |                                                                                   |
| Iris Data                  | LogisticRegression         | neg_log_loss           | Error: y_true contains only one   label (Iris-setosa). Please provide the true labels explicitly through the   labels argument. |                                                                                   |
| Iris Data                  | LinearDiscriminantAnalysis | accuracy               | Error: The priors do not sum to 1. Renormalizing                                                                                |                                                                                   |
| Iris Data                  | LinearDiscriminantAnalysis | neg_mean_squared_error | Error: The priors do not sum to 1. Renormalizing                                                                                |                                                                                   |
| Iris Data                  | LinearDiscriminantAnalysis | neg_log_loss           | Error: The priors do not sum to 1. Renormalizing                                                                                |                                                                                   |
| Pima Indians Diabetes Data | KNeighborsRegressor        | accuracy               | Error: Can't handle mix of binary and continuous                                                                                | Data Prep? (Pima, [KNeighborsRegressor, LinearRegression], accuracy)              |
| Pima Indians Diabetes Data | KNeighborsRegressor        | neg_mean_squared_error | -0.196342447                                                                                                                    |                                                                                   |
| Pima Indians Diabetes Data | KNeighborsRegressor        | neg_log_loss           | Error: 'KNeighborsRegressor' object has no attribute 'predict_proba'                                                            | NG: (KNeighborsRegressor, neg_log_loss)                                           |
| Pima Indians Diabetes Data | LinearRegression           | accuracy               | Error: Can't handle mix of binary and continuous                                                                                | Data Prep? (Pima, [KNeighborsRegressor, LinearRegression], accuracy)              |
| Pima Indians Diabetes Data | LinearRegression           | neg_mean_squared_error | -0.162812507                                                                                                                    |                                                                                   |
| Pima Indians Diabetes Data | LinearRegression           | neg_log_loss           | Error: 'LinearRegression' object has no attribute 'predict_proba'                                                               | Data Prep? Or Model-Scoring?([Boston, Pima], LinearRegression,   neg_log_loss)    |
| Pima Indians Diabetes Data | LogisticRegression         | accuracy               | 0.769514696                                                                                                                     |                                                                                   |
| Pima Indians Diabetes Data | LogisticRegression         | neg_mean_squared_error | -0.230485304                                                                                                                    |                                                                                   |
| Pima Indians Diabetes Data | LogisticRegression         | neg_log_loss           | -0.492545523                                                                                                                    |                                                                                   |
| Pima Indians Diabetes Data | LinearDiscriminantAnalysis | accuracy               | 0.773462064                                                                                                                     |                                                                                   |
| Pima Indians Diabetes Data | LinearDiscriminantAnalysis | neg_mean_squared_error | -0.226537936                                                                                                                    |                                                                                   |
| Pima Indians Diabetes Data | LinearDiscriminantAnalysis | neg_log_loss           | -0.48565533                                                                                                                     |                                                                                   |

H2O.ai Adventures in Artificial Intelligence (ai)

Background

Although I HAVE NOT thought about Artificial Intelligence, ai, since i was a student in Michael Arbib’s class studying for my M.S., when i became aware of H2O.ai.com, i decided it was time to jump in.🙂

The following will be a chronicle of my adventures.🙂

THIS IS A WORK IN PROGRESS

Big Data Hadoop vs Apache Spark

Downloads: (H2O vs Sparkling Water)

H2O.ai’s offerings, H2O and Sparkling Water, seemed to pose the question, “What Big Data platform should I choose, Hadoop or Apache Spark?” I have learned that they are not competitors. Katherine Noyes says in Infoworld,

“They do different things. … Hadoop is essentially a distributed data infrastructure … Spark, on the other hand, is a data-processing tool that operates on those distributed data collection”.

OK. But which of H2O.ai’s Downloads, only 2 when i started, should i choose to investigate? I picked Sparkling Water because of a page explaining the ai “Classification” Use Case.

Goal Install & RUN PySparkling

Here’s some notes for PySparkling installation on Windows 10.
Be prepared for (SysAdmin, SysAdmin, … more SysAAdmin)!

Install Apache Spark (to use PySpark)

  • Apache Spark needs to be installed first
    • Downloads\ApacheSpark\spark-1.6.2-bin-hadoop2.6
    • > echo %PYSPARK_PYTHON% == C:\Python27\python.exe
    • Test Run PySpark in the PySpark Shell
    • > cd Downloads\ApacheSpark\spark-1.6.2-bin-hadoop2.6
    • > .\bin\pyspark.cmd
    • Test with QuickStart N.B. Click the Python_Tab
    • RESULT: OK
    • Test Run PySpark as a Self-Contained Application
    • Test with Self-Contained Application N.B. Click the Python_Tab
    • RESULT: NO GOOD –

Self-Contained PySpark RESULT

Here’s the Self-Contained RESULT with NO MODIFICATIONS of the sys.path

"""SimpleApp.py"""
from pyspark import SparkContext

logFile = "YOUR_SPARK_HOME/README.md"  # Should be some file on your system
sc = SparkContext("local", "Simple App")
logData = sc.textFile(logFile).cache()

numAs = logData.filter(lambda s: 'a' in s).count()
numBs = logData.filter(lambda s: 'b' in s).count()

print("Lines with a: %i, lines with b: %i" % (numAs, numBs))
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-024ac6cc8be6> in <module>()
      1 """SimpleApp.py"""
----> 2 from pyspark import SparkContext
      3 
      4 logFile = "YOUR_SPARK_HOME/README.md"  # Should be some file on your system
      5 sc = SparkContext("local", "Simple App")

ImportError: No module named pyspark

sys.path Analysis – PySpark Shell vs Plain Python27

<br /># This is pyspark shell sys.path
pysparkShellSysPath = '''
C:\Users\joeco\AppData\Local\Temp\spark-a00a2ab4-63a0-404f-b607-5f34c4206e76\userFiles-5a9b86bf-a518-4fef-b4de-b42005b143d5
C:\Python27\lib\site-packages\pywebview-0.8.2-py2.7.egg
C:\Python27\lib\site-packages\ouimeaux-0.7.9.post0-py2.7.egg
C:\Python27\lib\site-packages\gevent_socketio-0.3.6-py2.7.egg
C:\Python27\lib\site-packages\flask_restful-0.3.5-py2.7.egg
C:\Python27\lib\site-packages\pysignals-0.1.2-py2.7.egg
C:\Python27\lib\site-packages\pyyaml-3.11-py2.7-win-amd64.egg
C:\Python27\lib\site-packages\requests-2.9.1-py2.7.egg
C:\Python27\lib\site-packages\gevent-1.1rc3-py2.7-win-amd64.egg
C:\Python27\lib\site-packages\gevent_websocket-0.9.5-py2.7.egg
C:\Python27\lib\site-packages\pytz-2015.7-py2.7.egg
C:\Python27\lib\site-packages\six-1.10.0-py2.7.egg
C:\Python27\lib\site-packages\flask-0.10.1-py2.7.egg
C:\Python27\lib\site-packages\aniso8601-1.1.0-py2.7.egg
C:\Python27\lib\site-packages\greenlet-0.4.9-py2.7-win-amd64.egg
C:\Python27\lib\site-packages\itsdangerous-0.24-py2.7.egg
C:\Python27\lib\site-packages\werkzeug-0.11.3-py2.7.egg
C:\Python27\lib\site-packages\python_dateutil-2.4.2-py2.7.egg
C:\Python27\lib\site-packages\python_registry-1.1.0-py2.7.egg
C:\Python27\lib\site-packages\enum34-1.1.2-py2.7.egg
C:\Python27\lib\site-packages\speedtest_cli-0.3.4-py2.7.egg
C:\Python27\lib\site-packages\midi-0.2.3-py2.7.egg
C:\Python27\lib\site-packages\h2o_pysparkling_1.6-1.6.5-py2.7.egg
C:\Python27\lib\site-packages\tabulate-0.7.5-py2.7.egg
C:\Python27\lib\site-packages\future-0.15.2-py2.7.egg
C:\Users\joeco\Downloads\ApacheSpark\spark-1.6.2-bin-hadoop2.6\python\lib\py4j-0.9-src.zip
C:\Users\joeco\Downloads\ApacheSpark\spark-1.6.2-bin-hadoop2.6\python
C:\Users\joeco\Downloads\ApacheSpark\spark-1.6.2-bin-hadoop2.6
C:\WINDOWS\SYSTEM32\python27.zip
C:\Python27\DLLs
C:\Python27\lib
C:\Python27\lib\plat-win
C:\Python27\lib\lib-tk
C:\Python27
C:\Python27\lib\site-packages
C:\Python27\lib\site-packages\win32
C:\Python27\lib\site-packages\win32\lib
C:\Python27\lib\site-packages\Pythonwin
C:\Python27\lib\site-packages\wx-3.0-msw
'''


# This is plain python 2.7 sys.path
plainPythonSysPath = '''
C:\Python27\lib\site-packages\pywebview-0.8.2-py2.7.egg
C:\Python27\lib\site-packages\ouimeaux-0.7.9.post0-py2.7.egg
C:\Python27\lib\site-packages\gevent_socketio-0.3.6-py2.7.egg
C:\Python27\lib\site-packages\flask_restful-0.3.5-py2.7.egg
C:\Python27\lib\site-packages\pysignals-0.1.2-py2.7.egg
C:\Python27\lib\site-packages\pyyaml-3.11-py2.7-win-amd64.egg
C:\Python27\lib\site-packages\requests-2.9.1-py2.7.egg
C:\Python27\lib\site-packages\gevent-1.1rc3-py2.7-win-amd64.egg
C:\Python27\lib\site-packages\gevent_websocket-0.9.5-py2.7.egg
C:\Python27\lib\site-packages\pytz-2015.7-py2.7.egg
C:\Python27\lib\site-packages\six-1.10.0-py2.7.egg
C:\Python27\lib\site-packages\flask-0.10.1-py2.7.egg
C:\Python27\lib\site-packages\aniso8601-1.1.0-py2.7.egg
C:\Python27\lib\site-packages\greenlet-0.4.9-py2.7-win-amd64.egg
C:\Python27\lib\site-packages\itsdangerous-0.24-py2.7.egg
C:\Python27\lib\site-packages\werkzeug-0.11.3-py2.7.egg
C:\Python27\lib\site-packages\python_dateutil-2.4.2-py2.7.egg
C:\Python27\lib\site-packages\python_registry-1.1.0-py2.7.egg
C:\Python27\lib\site-packages\enum34-1.1.2-py2.7.egg
C:\Python27\lib\site-packages\speedtest_cli-0.3.4-py2.7.egg
C:\Python27\lib\site-packages\midi-0.2.3-py2.7.egg
C:\Python27\lib\site-packages\h2o_pysparkling_1.6-1.6.5-py2.7.egg
C:\Python27\lib\site-packages\tabulate-0.7.5-py2.7.egg
C:\Python27\lib\site-packages\future-0.15.2-py2.7.egg
C:\WINDOWS\SYSTEM32\python27.zip
C:\Python27\DLLs
C:\Python27\lib
C:\Python27\lib\plat-win
C:\Python27\lib\lib-tk
C:\Python27
C:\Python27\lib\site-packages
C:\Python27\lib\site-packages\win32
C:\Python27\lib\site-packages\win32\lib
C:\Python27\lib\site-packages\Pythonwin
C:\Python27\lib\site-packages\wx-3.0-msw
'''

# print('hello', len( sorted(pysprkShellSysPath.splitlines()) ) )


pysparksp = sorted(pysparkShellSysPath.splitlines())
len(pysparksp)
plainsp = sorted(plainPythonSysPath.splitlines())
len(plainsp)
for i in range( max(len(pysparksp),len(plainsp)) ):
    if i < len(pysparksp):  print ('pyspk', pysparksp[i])
    if i < len(plainsp  ):  print ('plain', plainsp[i])
    print 

----------------------OUTPUT---------------------------
('pyspk', '')
('plain', '')

('pyspk', 'C:\\Python27')
('plain', 'C:\\Python27')

('pyspk', 'C:\\Python27\\DLLs')
('plain', 'C:\\Python27\\DLLs')

('pyspk', 'C:\\Python27\\lib')
('plain', 'C:\\Python27\\lib')

('pyspk', 'C:\\Python27\\lib\\lib-tk')
('plain', 'C:\\Python27\\lib\\lib-tk')

('pyspk', 'C:\\Python27\\lib\\plat-win')
('plain', 'C:\\Python27\\lib\\plat-win')

('pyspk', 'C:\\Python27\\lib\\site-packages')
('plain', 'C:\\Python27\\lib\\site-packages')

('pyspk', 'C:\\Python27\\lib\\site-packages')
('plain', 'C:\\Python27\\lib\\site-packages')

('pyspk', 'C:\\Python27\\lib\\site-packages\x07niso8601-1.1.0-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\x07niso8601-1.1.0-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\tabulate-0.7.5-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\tabulate-0.7.5-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\x0clask-0.10.1-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\x0clask-0.10.1-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\x0clask_restful-0.3.5-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\x0clask_restful-0.3.5-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\x0cuture-0.15.2-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\x0cuture-0.15.2-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\Pythonwin')
('plain', 'C:\\Python27\\lib\\site-packages\\Pythonwin')

('pyspk', 'C:\\Python27\\lib\\site-packages\\enum34-1.1.2-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\enum34-1.1.2-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\gevent-1.1rc3-py2.7-win-amd64.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\gevent-1.1rc3-py2.7-win-amd64.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\gevent_socketio-0.3.6-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\gevent_socketio-0.3.6-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\gevent_websocket-0.9.5-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\gevent_websocket-0.9.5-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\greenlet-0.4.9-py2.7-win-amd64.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\greenlet-0.4.9-py2.7-win-amd64.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\h2o_pysparkling_1.6-1.6.5-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\h2o_pysparkling_1.6-1.6.5-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\itsdangerous-0.24-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\itsdangerous-0.24-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\midi-0.2.3-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\midi-0.2.3-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\ouimeaux-0.7.9.post0-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\ouimeaux-0.7.9.post0-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\pysignals-0.1.2-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\pysignals-0.1.2-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\python_dateutil-2.4.2-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\python_dateutil-2.4.2-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\python_registry-1.1.0-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\python_registry-1.1.0-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\pytz-2015.7-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\pytz-2015.7-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\pywebview-0.8.2-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\pywebview-0.8.2-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\pyyaml-3.11-py2.7-win-amd64.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\pyyaml-3.11-py2.7-win-amd64.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\six-1.10.0-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\six-1.10.0-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\speedtest_cli-0.3.4-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\speedtest_cli-0.3.4-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\werkzeug-0.11.3-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\werkzeug-0.11.3-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\win32')
('plain', 'C:\\Python27\\lib\\site-packages\\win32')

('pyspk', 'C:\\Python27\\lib\\site-packages\\win32\\lib')
('plain', 'C:\\Python27\\lib\\site-packages\\win32\\lib')

('pyspk', 'C:\\Python27\\lib\\site-packages\\wx-3.0-msw')
('plain', 'C:\\Python27\\lib\\site-packages\\wx-3.0-msw')

('pyspk', 'C:\\Users\\joeco\\AppData\\Local\\Temp\\spark-a00a2ab4-63a0-404f-b607-5f34c4206e76\\userFiles-5a9b86bf-a518-4fef-b4de-b42005b143d5')
('plain', 'C:\\WINDOWS\\SYSTEM32\\python27.zip')

('pyspk', 'C:\\Users\\joeco\\Downloads\\ApacheSpark\\spark-1.6.2-bin-hadoop2.6')
('plain', 'equests-2.9.1-py2.7.egg')

('pyspk', 'C:\\Users\\joeco\\Downloads\\ApacheSpark\\spark-1.6.2-bin-hadoop2.6\\python')

('pyspk', 'C:\\Users\\joeco\\Downloads\\ApacheSpark\\spark-1.6.2-bin-hadoop2.6\\python\\lib\\py4j-0.9-src.zip')

('pyspk', 'C:\\WINDOWS\\SYSTEM32\\python27.zip')

('pyspk', 'equests-2.9.1-py2.7.egg')

My Question: Am I missing some secret sauce to set the sys.path?

Answer: I don’t know yet.

Pursuing the Secret, sys.path, Sauce

This morning i went back to my Original Goal, run PySparkling, NOT PySpark, but PySparkling.

  • I found a new download page for Sparkling Water, PySparkling’s Uncle?🙂
  • I chose my Spark version 1.6 out of [1.4, 1.5, 1.6]
  • Went to the 1.6 Download Page, downloaded Sparkling Water and clicked on the Python Tab which said “Get started with PySparkling”. Hallelulia!🙂

Get started with PySparkling Steps

  1. Download Spark
    1.1 DONE
  2. Point SPARK_HOME to the existing installation of Spark and export variable MASTER.
    >echo %SPARK_HOME%
    ...\Downloads\ApacheSpark\spark-1.6.2-bin-hadoop2.6
    
    >echo %MASTER%
    local-cluster[3,2,1024]
    
    >
    
  3. From your terminal, run:
    #To start an interactive Python terminal-
    bin/pysparkling
    

PROBLEM: No bin/pysparkling.cmd

BUT sparkling-env.cmd exists

  • Could bin/pysparkling or bin/sparkling-env.cmd contain Secret sys.path Sauce?
  • I Need a break from Sys Admin
  • I Need TO CODE SOMETHING

MORE NEXT TIME!🙂

TODO – FOLLOWING NEEDS WORK = UNPUBLISHED

PySparkling Installation

N.B. Click the Python_Tab

  • Be careful of your python environment.
  • I am running 2 & 3.
  • PySparkling needs 2.
C:\Users\joeco>python
Python 3.5.1 (v3.5.1:37a07cee5969, Dec  6 2015, 01:54:25) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from pysparkling import Context
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python35\lib\site-packages\h2o_pysparkling_1.6-1.6.5-py3.5.egg\pysparkling\__init__.py", line 11, in <module>
    from pysparkling.context import H2OContext
  File "C:\Python35\lib\site-packages\h2o_pysparkling_1.6-1.6.5-py3.5.egg\pysparkling\context.py", line 142
    print self
             ^
SyntaxError: Missing parentheses in call to 'print'
>>>
  • Be careful of your version of Sparkling Water
  • Be careful of your version of Spark

Use Windows Explorer & Bash on Ubuntu on Windows

Question

I started by asking myself, “How can I use the Windows Explorer to create folders or files in my home directory using ‘Bash on Ubuntu on Windows’?”AskUbuntu

I tried the following:

Enter C:\Users{user}\AppData\Local\lxss{username} in the Windows Explorer
Create a folder named "scripts" using Windows Explorer
Open Bash on Ubuntu on Windows
$ ls -a
total 0

RESULT: total 0 DOESN’T WORK

I wondered if there any plans to make something like this work?

Work Around

I found “sort of” a Work Around

1. I used bash commands $ mkdir and $ touch to create folders and files 
i.e. scripts/menu.py
2. I used the Windows Explorer and Notepad++ to edit menu.py
3. $ python scripts/menu.py 
WORKS

Flask with pywebview Example

Flask with pywebview Example

I mashed up 2 things:

Here’s the project dir structure (minimal Flask):

C:\Users\joeco\PycharmProjects\pyWebviewFlask>tree /F /A
Folder PATH listing for volume Windows
Volume serial number is F2FB-4810
C:.
|   pyWebviewFlask.py
|
+---.idea
|       encodings.xml
|       misc.xml
|       modules.xml
|       pyWebviewFlask.iml
|       workspace.xml
|
+---static
\---templates

Here’s the code in pyWebviewFlask.py:

"""

__author__      = "Joe Dorocak (JoeCodeswell@gmail.com)"
__copyright__   = "Copyright 2016, Joe Dorocak (JoeCodeswell.com)"
__license__ = "MIT"

PyCharm project type: flask
Templating language (std for Flask) : Jinja2
pywebview == webview: https://github.com/r0x0r/pywebview (by Roman Sirokov already installed for py2 & py3)
    https://github.com/r0x0r/pywebview/blob/master/examples/http_server.py

Flask views: http://flask.pocoo.org/docs/0.11/tutorial/views/

https://opensource.org/licenses

"""
from flask import Flask


import webview
import sys
import threading

app = Flask(__name__)


@app.route('/')
def hello_world():
    return 'Hello to the World!'

def start_server():
    app.run()

if __name__ == '__main__':
    """  https://github.com/r0x0r/pywebview/blob/master/examples/http_server.py
    """

    t = threading.Thread(target=start_server)
    t.daemon = True
    t.start()

    webview.create_window("It works, Joe!", "http://127.0.0.1:5000/")

    sys.exit()

Here’s a link to a screenshot of the result:
flask-pyview-shot

Simple Django Tutorial

Abstract

This post describes the steps required to generate a Multi-App Django Site (aka Project) that contains 2 simple Apps:

  1. Hello App – a Minimal App
  2. Contacts App

See JoeCodeswell Downloads – DjangoTutMysite.zip for a zip of the Final Code.

Content

Click here to go to content.

1. Prerequisites

  1. Installed Django 1.9+
  2. Above is Running on Python 3
  3. You need to understand the basics of urls and Views. See below.)

1.1 Brief basics of urls and Views

  1. A user enters a url into her browser.
  2. Django’s urls.py files try to match that url to a View.
  3. If a match is found the View returns the resulting HTML to the user’s browser.
    Here’s more from Django Girls
    See also, the IMPORTANT Django Doc “URL dispatcher“.

2. Create a Project i.e. a Site

N.B. Django conflates Project with Site.

2.1 Run django-admin startproject

Assume CODE_HOME stands for the directory where you’d like to store your code.

> cd CODE_HOME
> django-admin startproject mysite

Here’s the resulting file structure. N.B. The top directory, named mysite, CONTAINS a sub-directory with the SAME NAME, mysite.

CODE_HOME/
    mysite/
        manage.py
        mysite/
            __init__.py
            settings.py
            urls.py
            wsgi.py

2.2 Verify the Site

> cd CODE_HOME/mysite
> python manage.py runserver

You should see something like the following.

Performing system checks...

System check identified no issues (0 silenced).

[FOR THIS TUTORIAL, STUFF HERE, WAS DELETED.]

Django version 1.9.5, using settings 'mysite.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CTRL-BREAK.

Browsing to http://127.0.0.1:8000/ you should see something like the following.

It worked!
Congratulations on your first Django-powered page.

2.3 Run createsuperuser

Now we run the createsuperuser command so we can use a superuser account to do Admin functions.

> cd CODE_HOME/mysite
> python manage.py createsuperuser

Output will look like this.

Username (leave blank to use 'whatever'):
Email address: w@whatever.com
Password:
Password (again):
This password is too short. It must contain at least 8 characters.
Password:
Password (again):
Superuser created successfully.

3. Hello App

Let’s make a Hello App and call it “hello”.

N.B. We will run manage.py startapp NOT django-admin startproject.
Again, assume CODE_HOME stands for the directory where you “stored your code”.
Remember the resulting tree from startproject was as follows.

CODE_HOME/
    mysite/
        manage.py
        mysite/
            __init__.py
            settings.py
            urls.py
            wsgi.py

3.1 Run manage.py startapp

> cd CODE_HOME/mysite 
> python manage.py startapp hello

Here’s the resulting tree.

CODE_HOME/
    mysite/
        db.sqlite3
        manage.py
        hello/
            __init__.py          
            admin.py
            apps.py
            models.py
            tests.py
            views.py
            migrations/
                __init__.py    
        mysite/
            __init__.py
            settings.py
            urls.py
            wsgi.py

3.2 Add hello to settings.py

We need to edit the Site file, settings.py, to tell Django that our hello App exists. Add one line to CODE_HOME/mysite/mysite/settings.py as shown.

# CODE_HOME/mysite/mysite/settings.py

...
# Application definition

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'hello',                     # add this line
]
...

3.3 Add an App url

We need to add our hello App to our Site urls.py file.

After the deleted boilerplate change the code in CODE_HOME/mysite/mysite/urls.py to be as shown.

# CODE_HOME/mysite/mysite/urls.py

#[FOR THIS TUTORIAL, STUFF HERE, WAS DELETED.]

from django.conf.urls import include, url
from django.contrib import admin

urlpatterns = [
    url(r'^hello/', include('hello.urls', namespace='hello')), # added line
    url(r'^admin/', admin.site.urls),
]

The added line in urlpatterns, tells Django to send any url beginning with ‘hello’ to the hello App’s urls.py file for further processing.

We WILL CREATE the hello App’s urls.py file in the next step.

Here’s the Django Docs (1.9) for urls. django.conf.urls utility functions

3.4 Create our App’s urls.py file

In our Multi-App Site, each App will have its own urls.py file. We referenced that file in the include portion of the line we added to the Site urls.py file, in the previous step.

Now we need to create our hello App’s urls.py file. It will map an input url coming from a user’s browser, to a View.

Create the file CODE_HOME/mysite/hello/urls.py to be as follows.

# CODE_HOME/mysite/hello/urls.py
# This is the urls.py file for the hello App

from django.conf.urls import url
import hello.views

app_name = 'hello'
urlpatterns = [
    url(r'^$', hello.views.helloview, name='hello'), # View url    
]

The single line in urlpatterns defines the url for our View.

3.5 Create a View

Now we need to edit our hello App’s views.py file to create our View.

Edit CODE_HOME/mysite/hello/views.py.
Add lines under ‘# Create your views here.’, as shown.

from django.shortcuts import render

# Create your views here.

from django.http import HttpResponse

def helloview(request):
    return HttpResponse("Hello world!")

3.6 Test our View

If your server isn’t running already.

> cd CODE_HOME/mysite
> python manage.py runserver

In your browser, navigate to http://127.0.0.1:8000/hello/.
You should see:
Hello world!

4. Contacts App

Now let’s make a very SIMPLE Contacts App. We’ll call it ‘contacts’. It will implement CRUD functions on a database (DB). Django interfaces with a DB through Models.

We’ll follow many of the same steps we did when we made our Hello App. First, here’s the tree state of CODE_HOME, BEFORE we do anything and AFTER we made our Hello App.

. [== CODE_HOME]
+-- mysite
    +-- db.sqlite3
    +-- hello
    ¦   +-- __init__.py
    ¦   +-- __pycache__
            DELETED FOR BREVITY *.pyc files
    ¦   +-- admin.py
    ¦   +-- apps.py
    ¦   +-- migrations
    ¦   ¦   +-- __init__.py
    ¦   ¦   +-- __pycache__
                DELETED FOR BREVITY *.pyc files
    ¦   +-- models.py
    ¦   +-- tests.py
    ¦   +-- urls.py
    ¦   +-- views.py
    +-- manage.py
    +-- mysite
        +-- __init__.py
        +-- __pycache__
            DELETED FOR BREVITY *.pyc files
        +-- settings.py
        +-- urls.py
        +-- wsgi.py

4.1 Run manage.py startapp contacts

We did this BEFORE, to create our hello app. Now we’ll do it to create our contacts app

> cd CODE_HOME/mysite 
> python manage.py startapp contacts

Let’s inspect our tree state now.

. [== CODE_HOME]
+-- mysite
    +-- contacts
    ¦   +-- __init__.py
    ¦   +-- admin.py
    ¦   +-- apps.py
    ¦   +-- migrations
    ¦   ¦   +-- __init__.py
    ¦   +-- models.py
    ¦   +-- tests.py
    ¦   +-- views.py
    +-- db.sqlite3
    +-- hello
    ¦   +-- __init__.py
    ¦   +-- __pycache__
            DELETED FOR BREVITY *.pyc files
    ¦   +-- admin.py
    ¦   +-- apps.py
    ¦   +-- migrations
    ¦   ¦   +-- __init__.py
    ¦   ¦   +-- __pycache__
                DELETED FOR BREVITY *.pyc files
    ¦   +-- models.py
    ¦   +-- tests.py
    ¦   +-- urls.py
    ¦   +-- views.py
    +-- manage.py
    +-- mysite
        +-- __init__.py
        +-- __pycache__
            DELETED FOR BREVITY *.pyc files
        +-- settings.py
        +-- urls.py
        +-- wsgi.py

Notice there’s no __pycache__ dir under contacts yet. I think this is because we have not yet run this with our server.

4.2 Add contacts to settings.py like for hello

As for hello, we need to edit the Site file, settings.py, to tell Django that our contacts App exists. Add one line to CODE_HOME/mysite/mysite/settings.py as shown.

# CODE_HOME/mysite/mysite/settings.py

...
# Application definition

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'hello',                     # this line was added before
    'contacts',                  # add this line
]
...

4.3 Add an App url like for hello

As before for hello, we need to add our contacts App to our Site urls.py file.

Add the line in urlpatterns as shown.

# CODE_HOME/mysite/mysite/urls.py

#[FOR THIS TUTORIAL, STUFF HERE, WAS DELETED.]

from django.conf.urls import include, url
from django.contrib import admin

urlpatterns = [
    url(r'^contacts/', include('contacts.urls', namespace='contacts')), # added line
    url(r'^hello/', include('hello.urls', namespace='hello')), # line was added BEFORE
    url(r'^admin/', admin.site.urls),
]

4.4 Create our App’s urls.py file

As before, in our Multi-App Site, each App will have its own urls.py file. We referenced that file in the include portion of the line we added to the Site urls.py file, in the previous step.

Now we need to create our contacts App’s urls.py file. It will map an input url coming from a user’s browser, to a View.

Create the file CODE_HOME/mysite/contacts/urls.py to be as follows.

# CODE_HOME/mysite/contacts/urls.py
# This is the urls.py file for the contacts App

from django.conf.urls import url
import contacts.views

app_name = 'contacts'
urlpatterns = [
    url(r'^$', contacts.views.contactsview, name='contacts'), # View url    
]

The single line in urlpatterns defines the url for one of our Views.

4.5 Create a View

As before, we need to edit our contacts App’s views.py file to create our View.

Edit CODE_HOME/mysite/contacts/views.py.
Add lines under ‘# Create your views here.’, as shown.

from django.shortcuts import render

# Create your views here.

from django.http import HttpResponse

def contactsview(request):
    return HttpResponse("Hello world! from contacts")

4.6 Test our View

If your server isn’t running already.

> cd CODE_HOME/mysite
> python manage.py runserver

In your browser, navigate to http://127.0.0.1:8000/contacts/.
You should see:
Hello world! from contacts

4.7 Add a Model

Django Models are Classes that define database Tables.

We’re going to use the default (==SQLite) settings for our database. If you want to use a different db see Database setup.

Edit the existing file CODE_HOME/mysite/contacts/models.py

Add lines under ‘# Create your models here.’, as shown.

from django.db import models

# Create your models here.

class Contact(models.Model):
    first_name = models.CharField(max_length=255, )
    last_name = models.CharField(max_length=255, )
    email = models.EmailField()

    def __str__(self):
        return ' '.join([
            self.first_name,
            self.last_name,
            self.email,
        ])

    class Meta:
        app_label = 'contacts'

4.8 Run makemigrations

> cd CODE_HOME/mysite
> python manage.py makemigrations contacts

Output will look like this.

Migrations for 'contacts':
  0001_initial.py:
    - Create model Contact

We just created Migrations from our Models for our ‘contacts’ app.

4.9 Run migrate

Now, run migrate to use the Migrations to create the database Tables.

> cd CODE_HOME/mysite
> python manage.py migrate

Output will look similar to this.

Operations to perform:
  Apply all migrations: auth, contacts, admin, sessions, contenttypes
Running migrations:
  Rendering model states... DONE
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying contacts.0001_initial... OK
  Applying sessions.0001_initial... OK

4.10 Load the db using Admin

Now we’ll load the db with some Contacts using the Admin interface.
First, we have to tell Admin about our Contacts Model.

4.10.1 Register Model for Admin

Edit the existing file CODE_HOME/mysite/contacts/admin.py
Add lines under ‘# Register your models here.’, as shown.

from django.contrib import admin

# Register your models here.
from .models import Contact

admin.site.register(Contact)

Now our model will appear in our Admin interface.

4.10.2 Load the db

If your server isn’t running already.

> cd CODE_HOME/mysite
> python manage.py runserver

In your browser, navigate to http://127.0.0.1:8000/admin/.
Enter the Username & Password you entered when you ran createsuperuser. See above.

You should see a nicely formatted Django Admin webpage for your site. It should show something like the following.

Django administration …
Site administration …
Authentication and Authorization …
Contacts
Contacts +Add /Change

Click the +Add button and you should see the following representation of the Fields in our Contact model.

Add contact
First name:
Last name:
Email:

Fill in these three. We’ll use these for testing later.
– Joe Codeswell j@j.com
– Fred Belitnikoff fb@freddy.com
– Jayne Mansfield j@thebomb.com

4.11 List our existing contacts

We need to create a ListView for our contacts.
Then we need to create a Template for our ListView.
Then we need to hook up that view with our URLs.

4.11.1 Create a ListView

Let’s edit our contacts App’s views.py file to add our ListView.

Edit CODE_HOME/mysite/contacts/views.py.
Add lines under return HttpResponse("Hello world! from contacts"), as shown.

...

def contactsview(request):
    return HttpResponse("Hello world! from contacts")

# The new stuff is below here    
from django.views.generic import ListView
from contacts.models import Contact

class ListContactView(ListView):
    model = Contact
    template_name = 'contacts/contact_list.html'

4.11.2 Create a ListView Template

This will be the first Template we have created, since our Hello App didn’t use one.

4.11.2.1 Define Site Template Locations

First we need to determine where we will put the Templates for our site.
This is the subject of much discussion.
The following is my understanding of BEST PRACTICES for Django 1.9.
Edit CODE_HOME/mysite/mysite/settings.py.
– replace 'DIRS': [],
– with 'DIRS': [ *[os.path.join(BASE_DIR, os.path.join('templates', dir)) for dir in site_apps], ]

...

TEMPLATES = [
    {
    ...
        #'DIRS': [],
        'DIRS': [ 
                os.path.join(BASE_DIR, 'templates'),  
        ],

    ...

The BEST PRACTICES effects are:
1. To have all Templates for a Site under a SINGLE ROOT.
2. To keep App Templates separated under individual folders for NAMESPACE purposes.

4.11.2.2 Define our ListView Template

Create the following directory structure under CODE_HOME/mysite.

CODE_HOME
+---mysite
    +---templates
        +---contacts
        +---hello

Create the contact_list.html Template file in CODE_HOME/mysite/templates/contacts/ as follows.

<!-- CODE_HOME/mysite/templates/contacts/contact_list.html -->
<h1>Contacts</h1>

<ul>
  {% for contact in object_list %}
    <li class="contact">{{ contact }}</li>
  {% endfor %}
</ul>

4.11.3 Hook Up our ListView URL

Edit the existing file CODE_HOME/mysite/contacts/urls.py
Add the line shown to the urlpatterns List.

# CODE_HOME/mysite/contacts/urls.py

urlpatterns = [

    # add the following line
    url(r'^listview/$', contacts.views.ListContactView.as_view(), name='listview',),
    ...
]

4.11.4 Test our ListView URL

  1. If the server is NOT ALREADY running,
> cd CODE_HOME/mysite
> python manage.py runserver
  1. Browse to http://127.0.0.1:8000/contacts/listview/
  2. Expected Output is SOMETHING LIKE
Contacts

    Joe Codeswell j@j.com
    Fred Belitnikoff fb@freddy.com
    Jayne Mansfield j@thebomb.com

4.12 CRUD Contacts

CRUD Create, Read, Update and Delete are the primary database operations to be performed on our the elements in our Contacts model.
The Django Views that support CRUD functionality are called Generic Editing Views.
These Views support Making Queries to the DB.

So we can choose among the CRUD functions, we need to make a ListView enabling CRUD for each of our Contacts.

4.12.1 Make a CRUD List View

Edit CODE_HOME/mysite/contacts/views.py.
Add lines under our ListContactView definition as shown.

...

class ListContactView(ListView):
    model = Contact
    template_name = 'contacts/contact_list.html'

# new lines under here
class CRUDListView(ListView):
    model = Contact
    template_name = 'contacts/crudlist.html'

4.12.2 Make a CRUD List Template

Create the crudlist.html Template file in CODE_HOME/mysite/templates/contacts/ as follows.

<!-- CODE_HOME/mysite/templates/contacts/crudlist.html -->
<h1>Contacts</h1>
<a href="/contacts/create">Create</a>
<ul>
  {% for contact in object_list %}
    <li class="contact">{{ contact }} 
      <a href="/contacts/read/{{contact.id}}">Read | </a>
      <a href="/contacts/update/{{contact.id}}">Update | </a>
      <a href="/contacts/delete/{{contact.id}}">Delete</a>  
    </li>
  {% endfor %}
</ul>



4.12.3 Hook up the CRUD List URL

Edit the existing file CODE_HOME/mysite/contacts/urls.py
Add the line shown to the urlpatterns List.

# CODE_HOME/mysite/contacts/urls.py

urlpatterns = [

    # add the following line
    url(r'^crudlist/$', contacts.views.CRUDListView.as_view(), name='crudlist',),
    ...
]

4.12.4 Test CRUD List View

Navigate to http://127.0.0.1:8000/contacts/crudlist/
Expected result looks like the following.

Contacts
Create

    Joe Codeswell j@j.com Read | Update | Delete
    Fred Belitnikoff fb@freddy.com Read | Update | Delete
    Jayne Mansfield j@thebomb.com Read | Update | Delete

4.12.5 Make CRUD Create Contact

Let’s make & test our Create Contact — View, Template & URL Pattern.

View: Edit CODE_HOME/mysite/contacts/views.py.
Add lines under our existing CRUDListView definition as shown.

...

class CRUDListView(ListView):
    model = Contact
    template_name = 'contacts/crudlist.html'

# new lines under here
from django.core.urlresolvers import reverse
from django.views.generic import CreateView
class CreateContactView(CreateView):
    """ Inherits from GENERIC VIEW == CreateView """
    model = Contact
    template_name = 'contacts/create_contact.html'
    fields = ['first_name', 'last_name', 'email']

    def get_success_url(self):
        return reverse('contacts:list') #returns the URL of the Named URL, 'contacts list'

Template: Make the create_contact.html Template file in CODE_HOME/mysite/templates/contacts/ as follows.

<!-- CODE_HOME/mysite/templates/contacts/create_contact.html -->
<h1>Create Contact</h1>

<form action="{% url "contacts:create" %}" method="POST">
  {% csrf_token %}
  <ul>
    {{ form.as_ul }}
  </ul>
  <input id="save_contact" type="submit" value="Save" />
</form>

<a href="{% url "contacts:crudlist" %}">Back to CRUD list.</a>

URL Pattern: Add our Create Contact Url Pattern to CODE_HOME/mysite/contacts/urls.py as follows.

# CODE_HOME/mysite/contacts/urls.py
urlpatterns = [
    ...

    # add the following line
    url(r'^create/$', contacts.views.CreateContactView.as_view(), name='create',),

]

Test:
Navigate to http://127.0.0.1:8000/contacts/create/
Expected Result

Create Contact

    First name:
    Last name:
    Email:

4.12.6 Make CRUD Read Contact

Let’s make & test our Read Contact — View, Template & URL Pattern.
N.B. This Action works on a particular Contact selected by using our CRUD List View.
*View:** Edit CODE_HOME/mysite/contacts/views.py adding code after our CreateContactView class definition.

...
class CreateContactView(CreateView):
   ,,,
        return reverse('contacts:crudlist') #returns the URL of the Named URL, 'contacts list'

# added code under here
from django.views.generic import DetailView        
class ReadContactView(DetailView):
    model = Contact
    template_name = 'contacts/read_contact.html'        

Template: Make the read_contact.html Template file in CODE_HOME/mysite/templates/contacts/ as follows.

<!-- CODE_HOME/mysite/templates/contacts/read_contact.html -->

<h1>CRUD Read Detail</h1>

<p>ID: {{ object.id }}</p>
<p>Last Name: {{ object.last_name }}</p>
<p>First Name: {{ object.first_name }}</p>
<p>Email: {{ object.email }}</p>

URL Pattern: Add our Read Contact Url Pattern to CODE_HOME/mysite/contacts/urls.py as follows.

urlpatterns = [
    ...

    # add the following line
    url(r'^read/(?P<pk>[0-9]+)/$', contacts.views.ReadContactView.as_view(), name='read'),    
    # N.B. Note the use of the Regex Named Group, <pk>, defined special in Django to mean Primary Key.
...
]

N.B. See the note under the inserted url in the above code regarding the use of the Regex Named Group. We don’t even have to mention it Primary Key in our DetailView and Django uses it to Querry the DB for the matching item.

Test 1:
Navigate to http://127.0.0.1:8000/contacts/read/1/
Expected Result

CRUD Read Detail

ID: 1

Last Name: Codeswell

First Name: Joe

Email: j@j.com

Test 2:
Navigate to http://127.0.0.1:8010/contacts/crudlist/
Click on the Read link for the second item (“Fred Belitnikoff fb@freddy.com”)
Expected Result (Same Thing)

CRUD Read Detail

ID: 2

Last Name: Belitnikoff

First Name: Fred

Email: fb@freddy.com

4.12.7 Make CRUD Update Contact

Let’s make & test our Update Contact — View, Template & URL Pattern.
N.B. As with CRUD Read, this Action works on a particular Contact selected by using our CRUD List View.
View: Edit CODE_HOME/mysite/contacts/views.py adding code after our ReadContactView class definition.

...
class ReadContactView(DetailView):
    model = Contact
    template_name = 'contacts/read_contact.html'

# added code under here
from django.views.generic.edit import UpdateView
class UpdateContactView(UpdateView):
    model = Contact
    template_name = "contacts/update_contact.html"
    fields = ['last_name', 'first_name', 'email']
    success_url = 'contacts/crudlist/'

Template: Make the Update Form in the update_contact.html Template file in CODE_HOME/mysite/templates/contacts/ as follows.

<!-- CODE_HOME/mysite/templates/contacts/update_contact.html -->

<h1>Update Contact</h1>

<form action="" method="post">{% csrf_token %}
    {{ form.as_p }}
    <input type="submit" value="Update" />
    <button type="reset" value="Reset">Reset</button>
</form>
<button onclick="location.href='/contacts/crudlist'">Cancel</button>

URL Pattern: Add our Update Contact Url Pattern to CODE_HOME/mysite/contacts/urls.py as follows.

urlpatterns = [
    ...

    # add the following line
    url(r'^update/(?P<pk>[0-9]+)/$', contacts.views.UpdateContactView.as_view(), name='updatecontactview'),

    ...
]

Test 1:
A. Navigate to http://127.0.0.1:8000/contacts/update/1/
B. Change First name: Joe to Joseph
C. Press Update
Expected Result

  1. CRUD List is displayed.
  2. Joe Codeswell has been changed to Joseph Codeswell

Test 2:
A. Navigate to http://127.0.0.1:8000/contacts/crudlist/
B. Click on the Update link for Joseph Codeswell
C. Change First name: Joseph to Joe
D. Press Update
Expected Result

  1. CRUD List is displayed.
  2. Joseph Codeswell has been changed BACK to Joe Codeswell

Test 3:
A. Navigate to http://127.0.0.1:8000/contacts/crudlist/
B. Click on the Update link for Joseph Codeswell
C. Change First name: Joe to Joseph
D. Press Reset
Expected Result

  1. Update Contact is reset.
  2. Joseph has been RESET to Joe on the Update Contact form.

Test 4:
A. Navigate to http://127.0.0.1:8000/contacts/crudlist/
B. Click on the Update link for Joseph Codeswell
C. Change First name: Joe to Joseph
D. Press Cancel
Expected Result

  1. CRUD List is displayed.
  2. No change was made. Joe Codeswell STILL REMAINS LISTED as it was.

4.12.8 Make CRUD Delete Contact

Let’s make & test our Delete Contact — View, Template & URL Pattern.
Again, as with CRUD Read & Update, this Action works on a particular Contact selected by using our CRUD List View.
View: Edit CODE_HOME/mysite/contacts/views.py adding code after our UpdateContactView class definition.

...
class UpdateContactView(UpdateView):
   ...
# added code under here
from django.views.generic import DeleteView
class DeleteContactView(DeleteView):
    model = Contact
    template_name = "contacts/delete_contact.html"
    success_url = '/contacts/crudlist/'

Template: Make the Delete Form in the delete_contact.html Template file in CODE_HOME/mysite/templates/contacts/ as follows.

<!-- CODE_HOME/mysite/templates/contacts/delete_contact.html -->

<h1>Delete Contact</h1>

<form action="" method="post">{% csrf_token %}
    <p>Are you sure you want to delete ID: {{ object.id }}, "{{ object }}"?</p>
    <input type="submit" value="Confirm" />
</form>
<button onclick="location.href='/contacts/crudlist'">Cancel</button>

URL Pattern: Add our Delete Contact Url Pattern to CODE_HOME/mysite/contacts/urls.py as follows.

urlpatterns = [
    ...

    # add the following line
    url(r'^delete/(?P<pk>[0-9]+)/$', contacts.views.DeleteContactView.as_view(), name='deletecontactview'),

    ...
]

Test 1:
A. Navigate to http://127.0.0.1:8000/contacts/delete/1/
B. Press Cancel
Expected Result

  1. CRUD List is displayed.
  2. Joe Codeswell is STILL listed.

Test 2:
A. Navigate to http://127.0.0.1:8000/contacts/crudlist/
B. Click on the Delete link for Fred Belitnikoff
C. Press Confirm
Expected Result

  1. CRUD List is displayed.
  2. Fred Belitnikoff has been DELETED.

4.13 Search Contacts

We’ll search our Contacts with 2 types of searches:

  1. A Simple Search == SS (on first_name)
  2. A Parametric Search == PS

We’ll do one of the above for each type of Contact ListView == (ListContactView, CRUDListView) lets make

Yielding 4 Class Based Views:

  1. SimpleSearchListContactView(ListView)
  2. ParametricSearchListContactView(ListView)
  3. SimpleSearchCRUDListView(ListView)
  4. ParametricSearchCRUDListView(ListView)

4.13.1 Simple Search – ListContactView

In the Django Docs see:

  1. “Dynamic filtering”
  2. “Field lookups”

Let’s make & test our – SimpleSearchListContact — View, Template & URL Pattern.
View: Edit CODE_HOME/mysite/contacts/views.py adding code after our DeleteContactView class definition.

...
class DeleteContactView(DeleteView):
   ...
# added code under here
class SimpleSearchListContactView(ListView):

    model = Contact
    template_name = 'contacts/contact_list.html'    
    def get_queryset(self):
        qs = super(SimpleSearchListContactView, self).get_queryset()

        return qs.filter(first_name__contains=self.kwargs['fname'])
        # "first_name" is the first_name field in our Contacts model
        # "contains" is one of Django's "Field lookups"
        # "'fname'" is passed in from our url <fname> Regex Named Group  

Template: We reuse the Template from the ListContactView, contacts/contact_list.html. Nothing else to do here.
URL Pattern: Add our Simple Search ListContactView Url Pattern to CODE_HOME/mysite/contacts/urls.py as follows.

urlpatterns = [
    ...

    # add the following line
    url(r'^ssearch/(?P<fname>[\w-]+)/$', contacts.views.SimpleSearchListContactView.as_view(), name='simplesearchlistcontactview', ),
    # N.B. Note the use of the Regex Named Group, <fname>, sent to the SimpleSearchListContactView as a (name, value) pair used by self.kwargs.
    ...
]

Test 1:
Navigate to http://127.0.0.1:8000/contacts/ssearch/J/
Expected Result
Contact List is displayed showing:

Contacts
    Joe Codeswell j@j.com
    Jayne Mansfield j@thebomb.com

Test 2:
Navigate to http://127.0.0.1:8000/contacts/ssearch/a/
Expected Result
Contact List is displayed showing:

Contacts
    Jayne Mansfield j@thebomb.com

Test 3:
Navigate to http://127.0.0.1:8000/contacts/ssearch/o/
Expected Result
Contact List is displayed showing:

Contacts
    Joe Codeswell j@j.com

4.13.2 Parametric Search – ListContactView

Let’s make & test our – ParametricSearchListContact — View, Template & URL Pattern.
View: Edit CODE_HOME/mysite/contacts/views.py adding code after our SimpleSearchListContactView class definition.

...
class SimpleSearchListContactView(ListView):
   ...
# added code under here
class ParametricSearchListContactView(ListView):
    model = Contact
    template_name = 'contacts/contact_list.html'

    def get_queryset(self):
        qs = super(ParametricSearchListContactView, self).get_queryset()
        field_lookups_dict = {
            '%s__%s'%(self.kwargs['field_name'], self.kwargs['match_cmd']):self.kwargs['value'],
            # for a def of each match_cmd see Field lookups - https://docs.djangoproject.com/en/1.9/ref/models/querysets/#field-lookups
        }
        return qs.filter(**field_lookups_dict)

Template: We reuse the Template from the ListContactView, contacts/contact_list.html. Nothing else to do here.
URL Pattern: Add our Parametric Search ListContactView Url Pattern to CODE_HOME/mysite/contacts/urls.py as follows.

urlpatterns = [
    ...

    # add the following line
    url(r'^psearch/(?P<field_name>[\w-]+)/(?P<match_cmd>[\w-]+)/(?P<value>[\w-]+)/$', contacts.views.ParametricSearchListContactView.as_view(), name='parametricsearchlistcontactview', ),

    ...
]

Test 1:
Navigate to http://127.0.0.1:8000/contacts/psearch/first_name/contains/J/
Expected Result
Contact List is displayed showing:

Contacts
    Joe Codeswell j@j.com
    Jayne Mansfield j@thebomb.com

Test 2:
Navigate to http://127.0.0.1:8000/contacts/psearch/email/contains/bomb/
Expected Result
Contact List is displayed showing:

Contacts
    Jayne Mansfield j@thebomb.com

Test 3:
Navigate to http://127.0.0.1:8000/contacts/psearch/last_name/endswith/eld/
Expected Result
Contact List is displayed showing:

Contacts
    Jayne Mansfield j@thebomb.com

Test 4:
Navigate to http://127.0.0.1:8000/contacts/psearch/last_name/exact/Codeswell/
Expected Result
Contact List is displayed showing:

Contacts
    Joe Codeswell j@j.com

Test 5:
Navigate to http://127.0.0.1:8000/contacts/psearch/last_name/startswith/Man/
Expected Result
Contact List is displayed showing:

Contacts
    Jayne Mansfield j@thebomb.com

Test 6:
Navigate to http://127.0.0.1:8000/contacts/psearch/email/regex/bo/
Expected Result
Contact List is displayed showing:

Contacts
    Jayne Mansfield j@thebomb.com

Test 7:
Navigate to http://127.0.0.1:8000/contacts/psearch/last_name/regex/ode/
Expected Result
Contact List is displayed showing:

Contacts
    Joe Codeswell j@j.com

In summary, see the Django Docs “Field lookups” for a full list of match_cmds like
contains
endswith
exact
startswith
reges
– etc.

4.13.3 Searches (Both Simple & Parametric) for CRUD List View

The CRUD ListView Simple & Parametric Searches are PRETTY MUCH THE SAME AS those for the NON-CRUD ListView.
There are ONLY 2 differences:

  1. The template_name = ‘contacts/crudlist.html’ NOT ‘contacts/contact_list.html’.
  2. The class names – these occur in 2 places in each view.

Let’s make ** & **test BOTH our – SimpleSearchCRUDListContactView & ParametricSearchCRUDListContactView — Views, Templates & URL Patterns AT THE SAME TIME.
View: Edit CODE_HOME/mysite/contacts/views.py adding code for BOTH after our ParametricSearchListContactView class definition.

...
class ParametricSearchListContactView(ListView):
   ...
# added code under here (2 classes for Simple & Parametric Search CRUD ListViews)
class SimpleSearchCRUDListContactView(ListView):

    model = Contact
    template_name = 'contacts/crudlist.html'    

    def get_queryset(self):
        qs = super(SimpleSearchCRUDListContactView, self).get_queryset()

        return qs.filter(first_name__contains=self.kwargs['fname'])
        # "first_name" is the first_name field in our Contacts model
        # "contains" is one of Django's "Field lookups"
        # "'fname'" is passed in from our url <fname> Regex Named Group  


class ParametricSearchCRUDListContactView(ListView):
    model = Contact
    template_name = 'contacts/crudlist.html'


    def get_queryset(self):
        qs = super(ParametricSearchCRUDListContactView, self).get_queryset()
        field_lookups_dict = {
            '%s__%s'%(self.kwargs['field_name'], self.kwargs['match_cmd']):self.kwargs['value'],
            # for a def of each match_cmd see Field lookups - https://docs.djangoproject.com/en/1.9/ref/models/querysets/#field-lookups
        }

        return qs.filter(**field_lookups_dict)

N.B. These classes are the same as the previous 2 EXCEPT:

  1. template_name = ‘contacts/crudlist.html’ NOT ‘contacts/contact_list.html’
  2. Class names are new:
    • Lines 5 & 10 “SimpleSearchCRUDListContactView”
    • Lines 18 & 23 “ParametricSearchCRUDListContactView”

Template: We reuse the Template from the CRUDListView, contacts/crudlist.html. Nothing else to do here.
URL Patterns: Add BOTH our Simple & Parametric Search CRUDListView Url Patterns to CODE_HOME/mysite/contacts/urls.py as follows.

urlpatterns = [
    ...

    # add the following 2 lines
    url(r'^ssearchc/(?P<fname>[\w-]+)/$', contacts.views.SimpleSearchCRUDListContactView.as_view(template_name='contacts/crudlist.htm'), name='ssearchc', ),
    url(r'^psearchc/(?P<field_name>[\w-]+)/(?P<match_cmd>[\w-]+)/(?P<value>[\w-]+)/$', contacts.views.ParametricSearchCRUDListContactView.as_view(template_name='contacts/crudlist.htm'), name='psearchc', ),

    ...
]

Test our CRUD Searches
Perform the “same”

  • 3 tests we did for Simple Searches
  • 7 tests we did for Parametrics Searches
    EXCEPT

  • Navigate to …/contacts/ssearchc/,,, NOT …/contacts/ssearch/… AND

  • Navigate to …/contacts/psearchc/,,, NOT …/contacts/psearch/…

EXPECTED RESULTS
Same as before
EXCEPT items are listed with read, update & delete links.

Content

Click here to go back to Abstract (i.e. Top of Document)

1. Prerequisites
– 1.1 Brief basics of urls and Views
2. Create a Project i.e. a Site
– 2.1 Run django-admin startproject
– 2.2 Verify the Site
3. Hello App
– 3.1 Run manage.py startapp
– 3.2 Add hello to settings.py
– 3.3 Add an App url
– 3.4 Create our App’s urls.py file
– 3.5 Create a View
– 3.6 Test our View
4. Contacts App
– 4.1 Run manage.py startapp contacts
– 4.2 Add contacts to settings.py like for hello
– 4.3 Add an App url like for hello
– 4.4 Create our App’s urls.py file
– 4.5 Create a View
– 4.6 Test our View
– 4.7 Add a Model
– 4.8 Run makemigrations
– 4.9 Run migrate
– 4.10 Load the db using Admin
—- 4.10.1 Register Model for Admin
—- 4.10.2 Load the db
– 4.11 List our existing contacts
—- 4.11.1 Create a ListView
—- 4.11.2 Create a ListView Template
—— 4.11.2.1 Define Site Template Locations
—— 4.11.2.2 Define our ListView Template
—- 4.11.3 Hook Up our ListView URL
—- 4.11.4 Test our ListView URL
– 4.12 CRUD Contacts
—- 4.12.1 Make a CRUD List View
—- 4.12.2 Make a CRUD List Template
—- 4.12.3 Hook up the CRUD List URL
—- 4.12.4 Test CRUD List View
—- 4.12.5 Make CRUD Create Contact
—- 4.12.6 Make CRUD Read Contact
—- 4.12.7 Make CRUD Update Contact
—- 4.12.8 Make CRUD Delete Contact
– 4.13 Search Contacts
—- 4.13.1 Simple Search – ListContactView
—- 4.13.2 Parametric Search – ListContactView
—- 4.13.3 Searches (Both Simple & Parametric) for CRUD List View