Python code creates a database from .csv

Content

  1. Documentation
  2. Code

1. Documentation

csv2sqliteDB.py:
1. Creates an sqlite Database (from SQLITE_FILEPATH) then
2. Creates and Loads TABLE_NAME (adding id INTEGER PRIMARY KEY )
FROM CSV_FILEPATH (with header, NO id field necessary)
INTO DATABASE_FILEPATH
CREATING DATABASE_FILEPATH if it doesn’t exist.

Usage:   csv2sqliteDB.py TABLE_NAME CSV_FILEPATH  SQLITE_FILEPATH
Example: csv2sqliteDB.py contacts   contacts2.csv contactsDB2id.db

2. Code


#!/usr/bin/python
#! python3
#! python2
# -*- coding: utf-8 -*-
"""csv2sqliteDB.py Creates and Loads TABLE_NAME (adding id INTEGER PRIMARY KEY )
      from CSV_FILEPATH (with header, NO id field necessary)
      into DATABASE_FILEPATH
          creating DATABASE_FILEPATH if it doesn't exist

Usage:   csv2sqliteDB.py TABLE_NAME CSV_FILEPATH  SQLITE_FILEPATH
Example: csv2sqliteDB.py contacts   contacts2.csv contactsDB2id.db
"""

# see:
    # Importing a CSV file into a sqlite3 - http://stackoverflow.com/a/2888042/601770
    # csv file using python with headers intact - http://stackoverflow.com/a/3428633/601770

import os, sys, csv, sqlite3

def csv2sqliteDB(table_name, csv_filepath, database_filepath):
    conn, cursor = createDatabaseIfNotExist(database_filepath)
    nlines = sum(1 for line in open(csv_filepath)) # http://stackoverflow.com/a/36973958/601770
    with open(csv_filepath, 'r') as f:
        reader = csv.reader(f)
        headerL = reader.__next__()

        createTable(table_name, conn, cursor, headerL)

        loadTable(table_name, conn, cursor, headerL, reader, nlines)
    conn.close()

def createDatabaseIfNotExist(database_filepath):
    # if we error, we rollback automatically, else commit!
    with sqlite3.connect(database_filepath) as conn:
        cursor = conn.cursor()
        cursor.execute('SELECT SQLITE_VERSION()')
        data = cursor.fetchone()
        print('SQLite version:', data)
        return conn, cursor

def createTable(table_name, conn, cursor,  headerL):
    """
    :param table_name:
    :param conn:
    :param cursor:
    :param header:
    :return:
    """
    # http://stackoverflow.com/a/12432311/601770
    # http: // stackoverflow.com / a / 19730169 / 601770

    textL = len(headerL)*'TEXT'.split()
    def_str = ', '.join( ['id INTEGER PRIMARY KEY'] + [i[0]+' '+i[1] for i in list(zip(headerL, textL))] )
    script_str = """DROP TABLE IF EXISTS %s;
        CREATE TABLE %s (%s);
    """%(table_name, table_name, def_str) # checks to see if table exists and makes a fresh table.

    cursor.executescript(script_str)


def loadTable(table_name, conn, cursor, headerL, reader, nlines):
    insert_str_model = "INSERT INTO {table_name} ({col_names}) VALUES({q_marks});"
    col_names = ','.join(headerL)
    q_marks = ','.join(len(headerL)*'?'.split())

    insert_str = insert_str_model.format(table_name=table_name, col_names=col_names, q_marks=q_marks)

    # DO THE INSERTS
    line_count = 0
    for row in reader:
        line_count += 1
        if line_count%10 == 1: print('line %s of %s'%(line_count, nlines))
        cursor.execute(insert_str, row)
        conn.commit()


NUM_ARGS = 3
def main():
    args = sys.argv[1:]
    if len(args) != NUM_ARGS or "-h" in args or "--help" in args:
        print (__doc__)
        sys.exit(2)
    csv2sqliteDB(args[0], args[1], args[2])

if __name__ == '__main__':
    main()

Advertisements

Teach Kids Coding – Turtle, Python, IDLE

Advantage – Immediate Feedback

As the student repeatedly presses the Debug Control’s “Over” button, she will be able to see the Turtle execute the commands she has given it.

Content

  • Summary
  • Video Link
  • Technique Steps
  • Code

Summary

This post will demonstrate a technique for teaching kids (or anyone) coding, using Turtle Graphics, Python, Python IDLE, Python IDLE’s Debug Control and Statement Execution Stepping.

This technique should work for Windows, Mac, and Linux (e.g. Raspberry Pi).

Video Link

Here is a link to a YouTube Video for this post.

Technique Steps

  1. Open your File Manager
    • “File Explorer” in Windows
    • “Finder” in Mac
    • “File Manager” in Raspberry Pi
  2. Navigate to the folder [CodeFolderPath] in which you have your code to be used. Most File Managers let you copy the path to a command line.
  3. Open a Command Line Window
    • “Command Prompt” in Windows
    • “Terminal” in Mac and Raspberry Pi
  4. At the Command Line, change directory to [CodeFolderPath], whatever that is.
    • example:
    • $ cd ~/pythonPjs/MasonPjs
  5. At the Command Line, run IDLE.
    • $ idle
    • This will bring up the IDLE Shell Window.
  6. In the IDLE Shell Window’s menu bar, click File Open and open the appropriate code file.
    • This will bring up the IDLE Code Window.
  7. In the IDLE Shell Window’s menu bar, click Debug/Debugger.
    • This will bring up the IDLE Debug Control Window.
  8. In the Debug Control Window:
    • UnTick the Stack Checkbox.
    • Tick the Source Checkbox.
  9. Minimize the IDLE Shell, File Manager and Command Line Windows. We want the student to focus on the important windows.
    • IDLE Debug Control
    • IDLE Code
    • Turtle Graphics
  10. Position the Debug Control & IDLE Code windows so you’ll be able to see them AND the Turtle Graphics Window SIMULTANEOUSLY. I plan on Turtle Graphics Window to be on the RIGHT of the screen. So I put:
    • Debug Contol – Top Left of the screen.
    • IDLE Code – Bottom Left of the screen. I make it about the same WIDTH as the Debug Control Window.
  11. In the IDLE Code window click on File/Run/Run Module
    • RESIZE it to be above the IDLE Code Window.
    • ALSO Minimize the IDLE Shell Window that (annoyingly 🙂 ) popped up again.
  12. Click on Debug Control Window’s “Over” button until the Turtle Graphics window appears. NOTICE with each click of “Over”.
    • NOTICE with each click, A NEW LINE in the Code Window is highlighted.
    • NOTICE when forward() or left() commands are executed, the Turtle Graphics Window responds with the appropriate action.
    • NOTICE after the line A1pos = pos(); A2abs_pos = abs(pos()) has been executed, the Debug Control Window contains 2 lines under the “Locals”
    • A1pos – (200.00,0.00) or something like this value
    • A2abs_pos – 200.00 or something like this value
      • Keeping track of the current values of these items.
    • NOTICE that after we execute the if abs(pos()) < 1: statement, we loop back to just under the while True: statement.
  13. Click on "Over" for about 30 more times. NOTICE the Turtle Graphics window cooking along, OBEYING the commands in the Code Window. 🙂
  14. Right Click on the break Line & select "Set Breakpoint".
    • NOTICE the yellow background.
  15. In the Debug Control Window Click on the "Go" button. This will run until just before we execute the breakpoint line.
    • WATCH the Trutle Graphics window do its thing.
  16. Click "Over".
    • NOTICE we exit the while True: loop.
  17. Click "Over".
    • NOTICE the end_fill() statement colors the star yellow.
  18. Click "Over" one last time.
    • NOTICE we are DONE. 🙂

Code

Here is the code we used in the video.

from turtle import *
setup (width=500, height=500, startx=500)
color('red', 'yellow')
begin_fill()
while True:
    forward(200)
    left(170)
    A1pos = pos(); A2abs_pos = abs(pos()) 
    if abs(pos()) < 1:
        break
end_fill()
done()

H2O.ai Adventures in Artificial Intelligence (ai)

Background

Although I HAVE NOT thought about Artificial Intelligence, ai, since i was a student in Michael Arbib’s class studying for my M.S., when i became aware of H2O.ai.com, i decided it was time to jump in. 🙂

The following will be a chronicle of my adventures. 🙂

THIS IS A WORK IN PROGRESS

Big Data Hadoop vs Apache Spark

Downloads: (H2O vs Sparkling Water)

H2O.ai’s offerings, H2O and Sparkling Water, seemed to pose the question, “What Big Data platform should I choose, Hadoop or Apache Spark?” I have learned that they are not competitors. Katherine Noyes says in Infoworld,

“They do different things. … Hadoop is essentially a distributed data infrastructure … Spark, on the other hand, is a data-processing tool that operates on those distributed data collection”.

OK. But which of H2O.ai’s Downloads, only 2 when i started, should i choose to investigate? I picked Sparkling Water because of a page explaining the ai “Classification” Use Case.

Goal Install & RUN PySparkling

Here’s some notes for PySparkling installation on Windows 10.
Be prepared for (SysAdmin, SysAdmin, … more SysAAdmin)!

Install Apache Spark (to use PySpark)

  • Apache Spark needs to be installed first
    • Downloads\ApacheSpark\spark-1.6.2-bin-hadoop2.6
    • > echo %PYSPARK_PYTHON% == C:\Python27\python.exe
    • Test Run PySpark in the PySpark Shell
    • > cd Downloads\ApacheSpark\spark-1.6.2-bin-hadoop2.6
    • > .\bin\pyspark.cmd
    • Test with QuickStart N.B. Click the Python_Tab
    • RESULT: OK
    • Test Run PySpark as a Self-Contained Application
    • Test with Self-Contained Application N.B. Click the Python_Tab
    • RESULT: NO GOOD –

Self-Contained PySpark RESULT

Here’s the Self-Contained RESULT with NO MODIFICATIONS of the sys.path

"""SimpleApp.py"""
from pyspark import SparkContext

logFile = "YOUR_SPARK_HOME/README.md"  # Should be some file on your system
sc = SparkContext("local", "Simple App")
logData = sc.textFile(logFile).cache()

numAs = logData.filter(lambda s: 'a' in s).count()
numBs = logData.filter(lambda s: 'b' in s).count()

print("Lines with a: %i, lines with b: %i" % (numAs, numBs))
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-024ac6cc8be6> in <module>()
      1 """SimpleApp.py"""
----> 2 from pyspark import SparkContext
      3 
      4 logFile = "YOUR_SPARK_HOME/README.md"  # Should be some file on your system
      5 sc = SparkContext("local", "Simple App")

ImportError: No module named pyspark

sys.path Analysis – PySpark Shell vs Plain Python27

<br /># This is pyspark shell sys.path
pysparkShellSysPath = '''
C:\Users\joeco\AppData\Local\Temp\spark-a00a2ab4-63a0-404f-b607-5f34c4206e76\userFiles-5a9b86bf-a518-4fef-b4de-b42005b143d5
C:\Python27\lib\site-packages\pywebview-0.8.2-py2.7.egg
C:\Python27\lib\site-packages\ouimeaux-0.7.9.post0-py2.7.egg
C:\Python27\lib\site-packages\gevent_socketio-0.3.6-py2.7.egg
C:\Python27\lib\site-packages\flask_restful-0.3.5-py2.7.egg
C:\Python27\lib\site-packages\pysignals-0.1.2-py2.7.egg
C:\Python27\lib\site-packages\pyyaml-3.11-py2.7-win-amd64.egg
C:\Python27\lib\site-packages\requests-2.9.1-py2.7.egg
C:\Python27\lib\site-packages\gevent-1.1rc3-py2.7-win-amd64.egg
C:\Python27\lib\site-packages\gevent_websocket-0.9.5-py2.7.egg
C:\Python27\lib\site-packages\pytz-2015.7-py2.7.egg
C:\Python27\lib\site-packages\six-1.10.0-py2.7.egg
C:\Python27\lib\site-packages\flask-0.10.1-py2.7.egg
C:\Python27\lib\site-packages\aniso8601-1.1.0-py2.7.egg
C:\Python27\lib\site-packages\greenlet-0.4.9-py2.7-win-amd64.egg
C:\Python27\lib\site-packages\itsdangerous-0.24-py2.7.egg
C:\Python27\lib\site-packages\werkzeug-0.11.3-py2.7.egg
C:\Python27\lib\site-packages\python_dateutil-2.4.2-py2.7.egg
C:\Python27\lib\site-packages\python_registry-1.1.0-py2.7.egg
C:\Python27\lib\site-packages\enum34-1.1.2-py2.7.egg
C:\Python27\lib\site-packages\speedtest_cli-0.3.4-py2.7.egg
C:\Python27\lib\site-packages\midi-0.2.3-py2.7.egg
C:\Python27\lib\site-packages\h2o_pysparkling_1.6-1.6.5-py2.7.egg
C:\Python27\lib\site-packages\tabulate-0.7.5-py2.7.egg
C:\Python27\lib\site-packages\future-0.15.2-py2.7.egg
C:\Users\joeco\Downloads\ApacheSpark\spark-1.6.2-bin-hadoop2.6\python\lib\py4j-0.9-src.zip
C:\Users\joeco\Downloads\ApacheSpark\spark-1.6.2-bin-hadoop2.6\python
C:\Users\joeco\Downloads\ApacheSpark\spark-1.6.2-bin-hadoop2.6
C:\WINDOWS\SYSTEM32\python27.zip
C:\Python27\DLLs
C:\Python27\lib
C:\Python27\lib\plat-win
C:\Python27\lib\lib-tk
C:\Python27
C:\Python27\lib\site-packages
C:\Python27\lib\site-packages\win32
C:\Python27\lib\site-packages\win32\lib
C:\Python27\lib\site-packages\Pythonwin
C:\Python27\lib\site-packages\wx-3.0-msw
'''


# This is plain python 2.7 sys.path
plainPythonSysPath = '''
C:\Python27\lib\site-packages\pywebview-0.8.2-py2.7.egg
C:\Python27\lib\site-packages\ouimeaux-0.7.9.post0-py2.7.egg
C:\Python27\lib\site-packages\gevent_socketio-0.3.6-py2.7.egg
C:\Python27\lib\site-packages\flask_restful-0.3.5-py2.7.egg
C:\Python27\lib\site-packages\pysignals-0.1.2-py2.7.egg
C:\Python27\lib\site-packages\pyyaml-3.11-py2.7-win-amd64.egg
C:\Python27\lib\site-packages\requests-2.9.1-py2.7.egg
C:\Python27\lib\site-packages\gevent-1.1rc3-py2.7-win-amd64.egg
C:\Python27\lib\site-packages\gevent_websocket-0.9.5-py2.7.egg
C:\Python27\lib\site-packages\pytz-2015.7-py2.7.egg
C:\Python27\lib\site-packages\six-1.10.0-py2.7.egg
C:\Python27\lib\site-packages\flask-0.10.1-py2.7.egg
C:\Python27\lib\site-packages\aniso8601-1.1.0-py2.7.egg
C:\Python27\lib\site-packages\greenlet-0.4.9-py2.7-win-amd64.egg
C:\Python27\lib\site-packages\itsdangerous-0.24-py2.7.egg
C:\Python27\lib\site-packages\werkzeug-0.11.3-py2.7.egg
C:\Python27\lib\site-packages\python_dateutil-2.4.2-py2.7.egg
C:\Python27\lib\site-packages\python_registry-1.1.0-py2.7.egg
C:\Python27\lib\site-packages\enum34-1.1.2-py2.7.egg
C:\Python27\lib\site-packages\speedtest_cli-0.3.4-py2.7.egg
C:\Python27\lib\site-packages\midi-0.2.3-py2.7.egg
C:\Python27\lib\site-packages\h2o_pysparkling_1.6-1.6.5-py2.7.egg
C:\Python27\lib\site-packages\tabulate-0.7.5-py2.7.egg
C:\Python27\lib\site-packages\future-0.15.2-py2.7.egg
C:\WINDOWS\SYSTEM32\python27.zip
C:\Python27\DLLs
C:\Python27\lib
C:\Python27\lib\plat-win
C:\Python27\lib\lib-tk
C:\Python27
C:\Python27\lib\site-packages
C:\Python27\lib\site-packages\win32
C:\Python27\lib\site-packages\win32\lib
C:\Python27\lib\site-packages\Pythonwin
C:\Python27\lib\site-packages\wx-3.0-msw
'''

# print('hello', len( sorted(pysprkShellSysPath.splitlines()) ) )


pysparksp = sorted(pysparkShellSysPath.splitlines())
len(pysparksp)
plainsp = sorted(plainPythonSysPath.splitlines())
len(plainsp)
for i in range( max(len(pysparksp),len(plainsp)) ):
    if i < len(pysparksp):  print ('pyspk', pysparksp[i])
    if i < len(plainsp  ):  print ('plain', plainsp[i])
    print 

----------------------OUTPUT---------------------------
('pyspk', '')
('plain', '')

('pyspk', 'C:\\Python27')
('plain', 'C:\\Python27')

('pyspk', 'C:\\Python27\\DLLs')
('plain', 'C:\\Python27\\DLLs')

('pyspk', 'C:\\Python27\\lib')
('plain', 'C:\\Python27\\lib')

('pyspk', 'C:\\Python27\\lib\\lib-tk')
('plain', 'C:\\Python27\\lib\\lib-tk')

('pyspk', 'C:\\Python27\\lib\\plat-win')
('plain', 'C:\\Python27\\lib\\plat-win')

('pyspk', 'C:\\Python27\\lib\\site-packages')
('plain', 'C:\\Python27\\lib\\site-packages')

('pyspk', 'C:\\Python27\\lib\\site-packages')
('plain', 'C:\\Python27\\lib\\site-packages')

('pyspk', 'C:\\Python27\\lib\\site-packages\x07niso8601-1.1.0-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\x07niso8601-1.1.0-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\tabulate-0.7.5-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\tabulate-0.7.5-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\x0clask-0.10.1-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\x0clask-0.10.1-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\x0clask_restful-0.3.5-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\x0clask_restful-0.3.5-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\x0cuture-0.15.2-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\x0cuture-0.15.2-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\Pythonwin')
('plain', 'C:\\Python27\\lib\\site-packages\\Pythonwin')

('pyspk', 'C:\\Python27\\lib\\site-packages\\enum34-1.1.2-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\enum34-1.1.2-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\gevent-1.1rc3-py2.7-win-amd64.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\gevent-1.1rc3-py2.7-win-amd64.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\gevent_socketio-0.3.6-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\gevent_socketio-0.3.6-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\gevent_websocket-0.9.5-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\gevent_websocket-0.9.5-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\greenlet-0.4.9-py2.7-win-amd64.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\greenlet-0.4.9-py2.7-win-amd64.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\h2o_pysparkling_1.6-1.6.5-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\h2o_pysparkling_1.6-1.6.5-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\itsdangerous-0.24-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\itsdangerous-0.24-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\midi-0.2.3-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\midi-0.2.3-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\ouimeaux-0.7.9.post0-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\ouimeaux-0.7.9.post0-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\pysignals-0.1.2-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\pysignals-0.1.2-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\python_dateutil-2.4.2-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\python_dateutil-2.4.2-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\python_registry-1.1.0-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\python_registry-1.1.0-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\pytz-2015.7-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\pytz-2015.7-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\pywebview-0.8.2-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\pywebview-0.8.2-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\pyyaml-3.11-py2.7-win-amd64.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\pyyaml-3.11-py2.7-win-amd64.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\six-1.10.0-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\six-1.10.0-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\speedtest_cli-0.3.4-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\speedtest_cli-0.3.4-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\werkzeug-0.11.3-py2.7.egg')
('plain', 'C:\\Python27\\lib\\site-packages\\werkzeug-0.11.3-py2.7.egg')

('pyspk', 'C:\\Python27\\lib\\site-packages\\win32')
('plain', 'C:\\Python27\\lib\\site-packages\\win32')

('pyspk', 'C:\\Python27\\lib\\site-packages\\win32\\lib')
('plain', 'C:\\Python27\\lib\\site-packages\\win32\\lib')

('pyspk', 'C:\\Python27\\lib\\site-packages\\wx-3.0-msw')
('plain', 'C:\\Python27\\lib\\site-packages\\wx-3.0-msw')

('pyspk', 'C:\\Users\\joeco\\AppData\\Local\\Temp\\spark-a00a2ab4-63a0-404f-b607-5f34c4206e76\\userFiles-5a9b86bf-a518-4fef-b4de-b42005b143d5')
('plain', 'C:\\WINDOWS\\SYSTEM32\\python27.zip')

('pyspk', 'C:\\Users\\joeco\\Downloads\\ApacheSpark\\spark-1.6.2-bin-hadoop2.6')
('plain', 'equests-2.9.1-py2.7.egg')

('pyspk', 'C:\\Users\\joeco\\Downloads\\ApacheSpark\\spark-1.6.2-bin-hadoop2.6\\python')

('pyspk', 'C:\\Users\\joeco\\Downloads\\ApacheSpark\\spark-1.6.2-bin-hadoop2.6\\python\\lib\\py4j-0.9-src.zip')

('pyspk', 'C:\\WINDOWS\\SYSTEM32\\python27.zip')

('pyspk', 'equests-2.9.1-py2.7.egg')

My Question: Am I missing some secret sauce to set the sys.path?

Answer: I don’t know yet.

Pursuing the Secret, sys.path, Sauce

This morning i went back to my Original Goal, run PySparkling, NOT PySpark, but PySparkling.

  • I found a new download page for Sparkling Water, PySparkling’s Uncle? 🙂
  • I chose my Spark version 1.6 out of [1.4, 1.5, 1.6]
  • Went to the 1.6 Download Page, downloaded Sparkling Water and clicked on the Python Tab which said “Get started with PySparkling”. Hallelulia! 🙂

Get started with PySparkling Steps

  1. Download Spark
    1.1 DONE
  2. Point SPARK_HOME to the existing installation of Spark and export variable MASTER.
    >echo %SPARK_HOME%
    ...\Downloads\ApacheSpark\spark-1.6.2-bin-hadoop2.6
    
    >echo %MASTER%
    local-cluster[3,2,1024]
    
    >
    
  3. From your terminal, run:
    #To start an interactive Python terminal-
    bin/pysparkling
    

PROBLEM: No bin/pysparkling.cmd

BUT sparkling-env.cmd exists

  • Could bin/pysparkling or bin/sparkling-env.cmd contain Secret sys.path Sauce?
  • I Need a break from Sys Admin
  • I Need TO CODE SOMETHING

MORE NEXT TIME! 🙂

TODO – FOLLOWING NEEDS WORK = UNPUBLISHED

PySparkling Installation

N.B. Click the Python_Tab

  • Be careful of your python environment.
  • I am running 2 & 3.
  • PySparkling needs 2.
C:\Users\joeco>python
Python 3.5.1 (v3.5.1:37a07cee5969, Dec  6 2015, 01:54:25) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from pysparkling import Context
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python35\lib\site-packages\h2o_pysparkling_1.6-1.6.5-py3.5.egg\pysparkling\__init__.py", line 11, in <module>
    from pysparkling.context import H2OContext
  File "C:\Python35\lib\site-packages\h2o_pysparkling_1.6-1.6.5-py3.5.egg\pysparkling\context.py", line 142
    print self
             ^
SyntaxError: Missing parentheses in call to 'print'
>>>
  • Be careful of your version of Sparkling Water
  • Be careful of your version of Spark

Flask with pywebview Example

Flask with pywebview Example

I mashed up 2 things:

Here’s the project dir structure (minimal Flask):

C:\Users\joeco\PycharmProjects\pyWebviewFlask>tree /F /A
Folder PATH listing for volume Windows
Volume serial number is F2FB-4810
C:.
|   pyWebviewFlask.py
|
+---.idea
|       encodings.xml
|       misc.xml
|       modules.xml
|       pyWebviewFlask.iml
|       workspace.xml
|
+---static
\---templates

Here’s the code in pyWebviewFlask.py:

"""

__author__      = "Joe Dorocak (JoeCodeswell@gmail.com)"
__copyright__   = "Copyright 2016, Joe Dorocak (JoeCodeswell.com)"
__license__ = "MIT"

PyCharm project type: flask
Templating language (std for Flask) : Jinja2
pywebview == webview: https://github.com/r0x0r/pywebview (by Roman Sirokov already installed for py2 & py3)
    https://github.com/r0x0r/pywebview/blob/master/examples/http_server.py

Flask views: http://flask.pocoo.org/docs/0.11/tutorial/views/

https://opensource.org/licenses

"""
from flask import Flask


import webview
import sys
import threading

app = Flask(__name__)


@app.route('/')
def hello_world():
    return 'Hello to the World!'

def start_server():
    app.run()

if __name__ == '__main__':
    """  https://github.com/r0x0r/pywebview/blob/master/examples/http_server.py
    """

    t = threading.Thread(target=start_server)
    t.daemon = True
    t.start()

    webview.create_window("It works, Joe!", "http://127.0.0.1:5000/")

    sys.exit()

Here’s a link to a screenshot of the result:
flask-pyview-shot

Simple Django Tutorial

Abstract

This post describes the steps required to generate a Multi-App Django Site (aka Project) that contains 2 simple Apps:

  1. Hello App – a Minimal App
  2. Contacts App

See JoeCodeswell Downloads – DjangoTutMysite.zip for a zip of the Final Code.

Content

Click here to go to content.

1. Prerequisites

  1. Installed Django 1.9+
  2. Above is Running on Python 3
  3. You need to understand the basics of urls and Views. See below.)

1.1 Brief basics of urls and Views

  1. A user enters a url into her browser.
  2. Django’s urls.py files try to match that url to a View.
  3. If a match is found the View returns the resulting HTML to the user’s browser.
    Here’s more from Django Girls
    See also, the IMPORTANT Django Doc “URL dispatcher“.

2. Create a Project i.e. a Site

N.B. Django conflates Project with Site.

2.1 Run django-admin startproject

Assume CODE_HOME stands for the directory where you’d like to store your code.

> cd CODE_HOME
> django-admin startproject mysite

Here’s the resulting file structure. N.B. The top directory, named mysite, CONTAINS a sub-directory with the SAME NAME, mysite.

CODE_HOME/
    mysite/
        manage.py
        mysite/
            __init__.py
            settings.py
            urls.py
            wsgi.py

2.2 Verify the Site

> cd CODE_HOME/mysite
> python manage.py runserver

You should see something like the following.

Performing system checks...

System check identified no issues (0 silenced).

[FOR THIS TUTORIAL, STUFF HERE, WAS DELETED.]

Django version 1.9.5, using settings 'mysite.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CTRL-BREAK.

Browsing to http://127.0.0.1:8000/ you should see something like the following.

It worked!
Congratulations on your first Django-powered page.

2.3 Run createsuperuser

Now we run the createsuperuser command so we can use a superuser account to do Admin functions.

> cd CODE_HOME/mysite
> python manage.py createsuperuser

Output will look like this.

Username (leave blank to use 'whatever'):
Email address: w@whatever.com
Password:
Password (again):
This password is too short. It must contain at least 8 characters.
Password:
Password (again):
Superuser created successfully.

3. Hello App

Let’s make a Hello App and call it “hello”.

N.B. We will run manage.py startapp NOT django-admin startproject.
Again, assume CODE_HOME stands for the directory where you “stored your code”.
Remember the resulting tree from startproject was as follows.

CODE_HOME/
    mysite/
        manage.py
        mysite/
            __init__.py
            settings.py
            urls.py
            wsgi.py

3.1 Run manage.py startapp

> cd CODE_HOME/mysite 
> python manage.py startapp hello

Here’s the resulting tree.

CODE_HOME/
    mysite/
        db.sqlite3
        manage.py
        hello/
            __init__.py          
            admin.py
            apps.py
            models.py
            tests.py
            views.py
            migrations/
                __init__.py    
        mysite/
            __init__.py
            settings.py
            urls.py
            wsgi.py

3.2 Add hello to settings.py

We need to edit the Site file, settings.py, to tell Django that our hello App exists. Add one line to CODE_HOME/mysite/mysite/settings.py as shown.

# CODE_HOME/mysite/mysite/settings.py

...
# Application definition

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'hello',                     # add this line
]
...

3.3 Add an App url

We need to add our hello App to our Site urls.py file.

After the deleted boilerplate change the code in CODE_HOME/mysite/mysite/urls.py to be as shown.

# CODE_HOME/mysite/mysite/urls.py

#[FOR THIS TUTORIAL, STUFF HERE, WAS DELETED.]

from django.conf.urls import include, url
from django.contrib import admin

urlpatterns = [
    url(r'^hello/', include('hello.urls', namespace='hello')), # added line
    url(r'^admin/', admin.site.urls),
]

The added line in urlpatterns, tells Django to send any url beginning with ‘hello’ to the hello App’s urls.py file for further processing.

We WILL CREATE the hello App’s urls.py file in the next step.

Here’s the Django Docs (1.9) for urls. django.conf.urls utility functions

3.4 Create our App’s urls.py file

In our Multi-App Site, each App will have its own urls.py file. We referenced that file in the include portion of the line we added to the Site urls.py file, in the previous step.

Now we need to create our hello App’s urls.py file. It will map an input url coming from a user’s browser, to a View.

Create the file CODE_HOME/mysite/hello/urls.py to be as follows.

# CODE_HOME/mysite/hello/urls.py
# This is the urls.py file for the hello App

from django.conf.urls import url
import hello.views

app_name = 'hello'
urlpatterns = [
    url(r'^$', hello.views.helloview, name='hello'), # View url    
]

The single line in urlpatterns defines the url for our View.

3.5 Create a View

Now we need to edit our hello App’s views.py file to create our View.

Edit CODE_HOME/mysite/hello/views.py.
Add lines under ‘# Create your views here.’, as shown.

from django.shortcuts import render

# Create your views here.

from django.http import HttpResponse

def helloview(request):
    return HttpResponse("Hello world!")

3.6 Test our View

If your server isn’t running already.

> cd CODE_HOME/mysite
> python manage.py runserver

In your browser, navigate to http://127.0.0.1:8000/hello/.
You should see:
Hello world!

4. Contacts App

Now let’s make a very SIMPLE Contacts App. We’ll call it ‘contacts’. It will implement CRUD functions on a database (DB). Django interfaces with a DB through Models.

We’ll follow many of the same steps we did when we made our Hello App. First, here’s the tree state of CODE_HOME, BEFORE we do anything and AFTER we made our Hello App.

. [== CODE_HOME]
+-- mysite
    +-- db.sqlite3
    +-- hello
    ¦   +-- __init__.py
    ¦   +-- __pycache__
            DELETED FOR BREVITY *.pyc files
    ¦   +-- admin.py
    ¦   +-- apps.py
    ¦   +-- migrations
    ¦   ¦   +-- __init__.py
    ¦   ¦   +-- __pycache__
                DELETED FOR BREVITY *.pyc files
    ¦   +-- models.py
    ¦   +-- tests.py
    ¦   +-- urls.py
    ¦   +-- views.py
    +-- manage.py
    +-- mysite
        +-- __init__.py
        +-- __pycache__
            DELETED FOR BREVITY *.pyc files
        +-- settings.py
        +-- urls.py
        +-- wsgi.py

4.1 Run manage.py startapp contacts

We did this BEFORE, to create our hello app. Now we’ll do it to create our contacts app

> cd CODE_HOME/mysite 
> python manage.py startapp contacts

Let’s inspect our tree state now.

. [== CODE_HOME]
+-- mysite
    +-- contacts
    ¦   +-- __init__.py
    ¦   +-- admin.py
    ¦   +-- apps.py
    ¦   +-- migrations
    ¦   ¦   +-- __init__.py
    ¦   +-- models.py
    ¦   +-- tests.py
    ¦   +-- views.py
    +-- db.sqlite3
    +-- hello
    ¦   +-- __init__.py
    ¦   +-- __pycache__
            DELETED FOR BREVITY *.pyc files
    ¦   +-- admin.py
    ¦   +-- apps.py
    ¦   +-- migrations
    ¦   ¦   +-- __init__.py
    ¦   ¦   +-- __pycache__
                DELETED FOR BREVITY *.pyc files
    ¦   +-- models.py
    ¦   +-- tests.py
    ¦   +-- urls.py
    ¦   +-- views.py
    +-- manage.py
    +-- mysite
        +-- __init__.py
        +-- __pycache__
            DELETED FOR BREVITY *.pyc files
        +-- settings.py
        +-- urls.py
        +-- wsgi.py

Notice there’s no __pycache__ dir under contacts yet. I think this is because we have not yet run this with our server.

4.2 Add contacts to settings.py like for hello

As for hello, we need to edit the Site file, settings.py, to tell Django that our contacts App exists. Add one line to CODE_HOME/mysite/mysite/settings.py as shown.

# CODE_HOME/mysite/mysite/settings.py

...
# Application definition

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'hello',                     # this line was added before
    'contacts',                  # add this line
]
...

4.3 Add an App url like for hello

As before for hello, we need to add our contacts App to our Site urls.py file.

Add the line in urlpatterns as shown.

# CODE_HOME/mysite/mysite/urls.py

#[FOR THIS TUTORIAL, STUFF HERE, WAS DELETED.]

from django.conf.urls import include, url
from django.contrib import admin

urlpatterns = [
    url(r'^contacts/', include('contacts.urls', namespace='contacts')), # added line
    url(r'^hello/', include('hello.urls', namespace='hello')), # line was added BEFORE
    url(r'^admin/', admin.site.urls),
]

4.4 Create our App’s urls.py file

As before, in our Multi-App Site, each App will have its own urls.py file. We referenced that file in the include portion of the line we added to the Site urls.py file, in the previous step.

Now we need to create our contacts App’s urls.py file. It will map an input url coming from a user’s browser, to a View.

Create the file CODE_HOME/mysite/contacts/urls.py to be as follows.

# CODE_HOME/mysite/contacts/urls.py
# This is the urls.py file for the contacts App

from django.conf.urls import url
import contacts.views

app_name = 'contacts'
urlpatterns = [
    url(r'^$', contacts.views.contactsview, name='contacts'), # View url    
]

The single line in urlpatterns defines the url for one of our Views.

4.5 Create a View

As before, we need to edit our contacts App’s views.py file to create our View.

Edit CODE_HOME/mysite/contacts/views.py.
Add lines under ‘# Create your views here.’, as shown.

from django.shortcuts import render

# Create your views here.

from django.http import HttpResponse

def contactsview(request):
    return HttpResponse("Hello world! from contacts")

4.6 Test our View

If your server isn’t running already.

> cd CODE_HOME/mysite
> python manage.py runserver

In your browser, navigate to http://127.0.0.1:8000/contacts/.
You should see:
Hello world! from contacts

4.7 Add a Model

Django Models are Classes that define database Tables.

We’re going to use the default (==SQLite) settings for our database. If you want to use a different db see Database setup.

Edit the existing file CODE_HOME/mysite/contacts/models.py

Add lines under ‘# Create your models here.’, as shown.

from django.db import models

# Create your models here.

class Contact(models.Model):
    first_name = models.CharField(max_length=255, )
    last_name = models.CharField(max_length=255, )
    email = models.EmailField()

    def __str__(self):
        return ' '.join([
            self.first_name,
            self.last_name,
            self.email,
        ])

    class Meta:
        app_label = 'contacts'

4.8 Run makemigrations

> cd CODE_HOME/mysite
> python manage.py makemigrations contacts

Output will look like this.

Migrations for 'contacts':
  0001_initial.py:
    - Create model Contact

We just created Migrations from our Models for our ‘contacts’ app.

4.9 Run migrate

Now, run migrate to use the Migrations to create the database Tables.

> cd CODE_HOME/mysite
> python manage.py migrate

Output will look similar to this.

Operations to perform:
  Apply all migrations: auth, contacts, admin, sessions, contenttypes
Running migrations:
  Rendering model states... DONE
  Applying contenttypes.0001_initial... OK
  Applying auth.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying contacts.0001_initial... OK
  Applying sessions.0001_initial... OK

4.10 Load the db using Admin

Now we’ll load the db with some Contacts using the Admin interface.
First, we have to tell Admin about our Contacts Model.

4.10.1 Register Model for Admin

Edit the existing file CODE_HOME/mysite/contacts/admin.py
Add lines under ‘# Register your models here.’, as shown.

from django.contrib import admin

# Register your models here.
from .models import Contact

admin.site.register(Contact)

Now our model will appear in our Admin interface.

4.10.2 Load the db

If your server isn’t running already.

> cd CODE_HOME/mysite
> python manage.py runserver

In your browser, navigate to http://127.0.0.1:8000/admin/.
Enter the Username & Password you entered when you ran createsuperuser. See above.

You should see a nicely formatted Django Admin webpage for your site. It should show something like the following.

Django administration …
Site administration …
Authentication and Authorization …
Contacts
Contacts +Add /Change

Click the +Add button and you should see the following representation of the Fields in our Contact model.

Add contact
First name:
Last name:
Email:

Fill in these three. We’ll use these for testing later.
– Joe Codeswell j@j.com
– Fred Belitnikoff fb@freddy.com
– Jayne Mansfield j@thebomb.com

4.11 List our existing contacts

We need to create a ListView for our contacts.
Then we need to create a Template for our ListView.
Then we need to hook up that view with our URLs.

4.11.1 Create a ListView

Let’s edit our contacts App’s views.py file to add our ListView.

Edit CODE_HOME/mysite/contacts/views.py.
Add lines under return HttpResponse("Hello world! from contacts"), as shown.

...

def contactsview(request):
    return HttpResponse("Hello world! from contacts")

# The new stuff is below here    
from django.views.generic import ListView
from contacts.models import Contact

class ListContactView(ListView):
    model = Contact
    template_name = 'contacts/contact_list.html'

4.11.2 Create a ListView Template

This will be the first Template we have created, since our Hello App didn’t use one.

4.11.2.1 Define Site Template Locations

First we need to determine where we will put the Templates for our site.
This is the subject of much discussion.
The following is my understanding of BEST PRACTICES for Django 1.9.
Edit CODE_HOME/mysite/mysite/settings.py.
– replace 'DIRS': [],
– with 'DIRS': [ *[os.path.join(BASE_DIR, os.path.join('templates', dir)) for dir in site_apps], ]

...

TEMPLATES = [
    {
    ...
        #'DIRS': [],
        'DIRS': [ 
                os.path.join(BASE_DIR, 'templates'),  
        ],

    ...

The BEST PRACTICES effects are:
1. To have all Templates for a Site under a SINGLE ROOT.
2. To keep App Templates separated under individual folders for NAMESPACE purposes.

4.11.2.2 Define our ListView Template

Create the following directory structure under CODE_HOME/mysite.

CODE_HOME
+---mysite
    +---templates
        +---contacts
        +---hello

Create the contact_list.html Template file in CODE_HOME/mysite/templates/contacts/ as follows.

<!-- CODE_HOME/mysite/templates/contacts/contact_list.html -->
<h1>Contacts</h1>

<ul>
  {% for contact in object_list %}
    <li class="contact">{{ contact }}</li>
  {% endfor %}
</ul>

4.11.3 Hook Up our ListView URL

Edit the existing file CODE_HOME/mysite/contacts/urls.py
Add the line shown to the urlpatterns List.

# CODE_HOME/mysite/contacts/urls.py

urlpatterns = [

    # add the following line
    url(r'^listview/$', contacts.views.ListContactView.as_view(), name='listview',),
    ...
]

4.11.4 Test our ListView URL

  1. If the server is NOT ALREADY running,
> cd CODE_HOME/mysite
> python manage.py runserver
  1. Browse to http://127.0.0.1:8000/contacts/listview/
  2. Expected Output is SOMETHING LIKE
Contacts

    Joe Codeswell j@j.com
    Fred Belitnikoff fb@freddy.com
    Jayne Mansfield j@thebomb.com

4.12 CRUD Contacts

CRUD Create, Read, Update and Delete are the primary database operations to be performed on our the elements in our Contacts model.
The Django Views that support CRUD functionality are called Generic Editing Views.
These Views support Making Queries to the DB.

So we can choose among the CRUD functions, we need to make a ListView enabling CRUD for each of our Contacts.

4.12.1 Make a CRUD List View

Edit CODE_HOME/mysite/contacts/views.py.
Add lines under our ListContactView definition as shown.

...

class ListContactView(ListView):
    model = Contact
    template_name = 'contacts/contact_list.html'

# new lines under here
class CRUDListView(ListView):
    model = Contact
    template_name = 'contacts/crudlist.html'

4.12.2 Make a CRUD List Template

Create the crudlist.html Template file in CODE_HOME/mysite/templates/contacts/ as follows.

<!-- CODE_HOME/mysite/templates/contacts/crudlist.html -->
<h1>Contacts</h1>
<a href="/contacts/create">Create</a>
<ul>
  {% for contact in object_list %}
    <li class="contact">{{ contact }} 
      <a href="/contacts/read/{{contact.id}}">Read | </a>
      <a href="/contacts/update/{{contact.id}}">Update | </a>
      <a href="/contacts/delete/{{contact.id}}">Delete</a>  
    </li>
  {% endfor %}
</ul>



4.12.3 Hook up the CRUD List URL

Edit the existing file CODE_HOME/mysite/contacts/urls.py
Add the line shown to the urlpatterns List.

# CODE_HOME/mysite/contacts/urls.py

urlpatterns = [

    # add the following line
    url(r'^crudlist/$', contacts.views.CRUDListView.as_view(), name='crudlist',),
    ...
]

4.12.4 Test CRUD List View

Navigate to http://127.0.0.1:8000/contacts/crudlist/
Expected result looks like the following.

Contacts
Create

    Joe Codeswell j@j.com Read | Update | Delete
    Fred Belitnikoff fb@freddy.com Read | Update | Delete
    Jayne Mansfield j@thebomb.com Read | Update | Delete

4.12.5 Make CRUD Create Contact

Let’s make & test our Create Contact — View, Template & URL Pattern.

View: Edit CODE_HOME/mysite/contacts/views.py.
Add lines under our existing CRUDListView definition as shown.

...

class CRUDListView(ListView):
    model = Contact
    template_name = 'contacts/crudlist.html'

# new lines under here
from django.core.urlresolvers import reverse
from django.views.generic import CreateView
class CreateContactView(CreateView):
    """ Inherits from GENERIC VIEW == CreateView """
    model = Contact
    template_name = 'contacts/create_contact.html'
    fields = ['first_name', 'last_name', 'email']

    def get_success_url(self):
        return reverse('contacts:list') #returns the URL of the Named URL, 'contacts list'

Template: Make the create_contact.html Template file in CODE_HOME/mysite/templates/contacts/ as follows.

<!-- CODE_HOME/mysite/templates/contacts/create_contact.html -->
<h1>Create Contact</h1>

<form action="{% url "contacts:create" %}" method="POST">
  {% csrf_token %}
  <ul>
    {{ form.as_ul }}
  </ul>
  <input id="save_contact" type="submit" value="Save" />
</form>

<a href="{% url "contacts:crudlist" %}">Back to CRUD list.</a>

URL Pattern: Add our Create Contact Url Pattern to CODE_HOME/mysite/contacts/urls.py as follows.

# CODE_HOME/mysite/contacts/urls.py
urlpatterns = [
    ...

    # add the following line
    url(r'^create/$', contacts.views.CreateContactView.as_view(), name='create',),

]

Test:
Navigate to http://127.0.0.1:8000/contacts/create/
Expected Result

Create Contact

    First name:
    Last name:
    Email:

4.12.6 Make CRUD Read Contact

Let’s make & test our Read Contact — View, Template & URL Pattern.
N.B. This Action works on a particular Contact selected by using our CRUD List View.
*View:** Edit CODE_HOME/mysite/contacts/views.py adding code after our CreateContactView class definition.

...
class CreateContactView(CreateView):
   ,,,
        return reverse('contacts:crudlist') #returns the URL of the Named URL, 'contacts list'

# added code under here
from django.views.generic import DetailView        
class ReadContactView(DetailView):
    model = Contact
    template_name = 'contacts/read_contact.html'        

Template: Make the read_contact.html Template file in CODE_HOME/mysite/templates/contacts/ as follows.

<!-- CODE_HOME/mysite/templates/contacts/read_contact.html -->

<h1>CRUD Read Detail</h1>

<p>ID: {{ object.id }}</p>
<p>Last Name: {{ object.last_name }}</p>
<p>First Name: {{ object.first_name }}</p>
<p>Email: {{ object.email }}</p>

URL Pattern: Add our Read Contact Url Pattern to CODE_HOME/mysite/contacts/urls.py as follows.

urlpatterns = [
    ...

    # add the following line
    url(r'^read/(?P<pk>[0-9]+)/$', contacts.views.ReadContactView.as_view(), name='read'),    
    # N.B. Note the use of the Regex Named Group, <pk>, defined special in Django to mean Primary Key.
...
]

N.B. See the note under the inserted url in the above code regarding the use of the Regex Named Group. We don’t even have to mention it Primary Key in our DetailView and Django uses it to Querry the DB for the matching item.

Test 1:
Navigate to http://127.0.0.1:8000/contacts/read/1/
Expected Result

CRUD Read Detail

ID: 1

Last Name: Codeswell

First Name: Joe

Email: j@j.com

Test 2:
Navigate to http://127.0.0.1:8010/contacts/crudlist/
Click on the Read link for the second item (“Fred Belitnikoff fb@freddy.com”)
Expected Result (Same Thing)

CRUD Read Detail

ID: 2

Last Name: Belitnikoff

First Name: Fred

Email: fb@freddy.com

4.12.7 Make CRUD Update Contact

Let’s make & test our Update Contact — View, Template & URL Pattern.
N.B. As with CRUD Read, this Action works on a particular Contact selected by using our CRUD List View.
View: Edit CODE_HOME/mysite/contacts/views.py adding code after our ReadContactView class definition.

...
class ReadContactView(DetailView):
    model = Contact
    template_name = 'contacts/read_contact.html'

# added code under here
from django.views.generic.edit import UpdateView
class UpdateContactView(UpdateView):
    model = Contact
    template_name = "contacts/update_contact.html"
    fields = ['last_name', 'first_name', 'email']
    success_url = 'contacts/crudlist/'

Template: Make the Update Form in the update_contact.html Template file in CODE_HOME/mysite/templates/contacts/ as follows.

<!-- CODE_HOME/mysite/templates/contacts/update_contact.html -->

<h1>Update Contact</h1>

<form action="" method="post">{% csrf_token %}
    {{ form.as_p }}
    <input type="submit" value="Update" />
    <button type="reset" value="Reset">Reset</button>
</form>
<button onclick="location.href='/contacts/crudlist'">Cancel</button>

URL Pattern: Add our Update Contact Url Pattern to CODE_HOME/mysite/contacts/urls.py as follows.

urlpatterns = [
    ...

    # add the following line
    url(r'^update/(?P<pk>[0-9]+)/$', contacts.views.UpdateContactView.as_view(), name='updatecontactview'),

    ...
]

Test 1:
A. Navigate to http://127.0.0.1:8000/contacts/update/1/
B. Change First name: Joe to Joseph
C. Press Update
Expected Result

  1. CRUD List is displayed.
  2. Joe Codeswell has been changed to Joseph Codeswell

Test 2:
A. Navigate to http://127.0.0.1:8000/contacts/crudlist/
B. Click on the Update link for Joseph Codeswell
C. Change First name: Joseph to Joe
D. Press Update
Expected Result

  1. CRUD List is displayed.
  2. Joseph Codeswell has been changed BACK to Joe Codeswell

Test 3:
A. Navigate to http://127.0.0.1:8000/contacts/crudlist/
B. Click on the Update link for Joseph Codeswell
C. Change First name: Joe to Joseph
D. Press Reset
Expected Result

  1. Update Contact is reset.
  2. Joseph has been RESET to Joe on the Update Contact form.

Test 4:
A. Navigate to http://127.0.0.1:8000/contacts/crudlist/
B. Click on the Update link for Joseph Codeswell
C. Change First name: Joe to Joseph
D. Press Cancel
Expected Result

  1. CRUD List is displayed.
  2. No change was made. Joe Codeswell STILL REMAINS LISTED as it was.

4.12.8 Make CRUD Delete Contact

Let’s make & test our Delete Contact — View, Template & URL Pattern.
Again, as with CRUD Read & Update, this Action works on a particular Contact selected by using our CRUD List View.
View: Edit CODE_HOME/mysite/contacts/views.py adding code after our UpdateContactView class definition.

...
class UpdateContactView(UpdateView):
   ...
# added code under here
from django.views.generic import DeleteView
class DeleteContactView(DeleteView):
    model = Contact
    template_name = "contacts/delete_contact.html"
    success_url = '/contacts/crudlist/'

Template: Make the Delete Form in the delete_contact.html Template file in CODE_HOME/mysite/templates/contacts/ as follows.

<!-- CODE_HOME/mysite/templates/contacts/delete_contact.html -->

<h1>Delete Contact</h1>

<form action="" method="post">{% csrf_token %}
    <p>Are you sure you want to delete ID: {{ object.id }}, "{{ object }}"?</p>
    <input type="submit" value="Confirm" />
</form>
<button onclick="location.href='/contacts/crudlist'">Cancel</button>

URL Pattern: Add our Delete Contact Url Pattern to CODE_HOME/mysite/contacts/urls.py as follows.

urlpatterns = [
    ...

    # add the following line
    url(r'^delete/(?P<pk>[0-9]+)/$', contacts.views.DeleteContactView.as_view(), name='deletecontactview'),

    ...
]

Test 1:
A. Navigate to http://127.0.0.1:8000/contacts/delete/1/
B. Press Cancel
Expected Result

  1. CRUD List is displayed.
  2. Joe Codeswell is STILL listed.

Test 2:
A. Navigate to http://127.0.0.1:8000/contacts/crudlist/
B. Click on the Delete link for Fred Belitnikoff
C. Press Confirm
Expected Result

  1. CRUD List is displayed.
  2. Fred Belitnikoff has been DELETED.

4.13 Search Contacts

We’ll search our Contacts with 2 types of searches:

  1. A Simple Search == SS (on first_name)
  2. A Parametric Search == PS

We’ll do one of the above for each type of Contact ListView == (ListContactView, CRUDListView) lets make

Yielding 4 Class Based Views:

  1. SimpleSearchListContactView(ListView)
  2. ParametricSearchListContactView(ListView)
  3. SimpleSearchCRUDListView(ListView)
  4. ParametricSearchCRUDListView(ListView)

4.13.1 Simple Search – ListContactView

In the Django Docs see:

  1. “Dynamic filtering”
  2. “Field lookups”

Let’s make & test our – SimpleSearchListContact — View, Template & URL Pattern.
View: Edit CODE_HOME/mysite/contacts/views.py adding code after our DeleteContactView class definition.

...
class DeleteContactView(DeleteView):
   ...
# added code under here
class SimpleSearchListContactView(ListView):

    model = Contact
    template_name = 'contacts/contact_list.html'    
    def get_queryset(self):
        qs = super(SimpleSearchListContactView, self).get_queryset()

        return qs.filter(first_name__contains=self.kwargs['fname'])
        # "first_name" is the first_name field in our Contacts model
        # "contains" is one of Django's "Field lookups"
        # "'fname'" is passed in from our url <fname> Regex Named Group  

Template: We reuse the Template from the ListContactView, contacts/contact_list.html. Nothing else to do here.
URL Pattern: Add our Simple Search ListContactView Url Pattern to CODE_HOME/mysite/contacts/urls.py as follows.

urlpatterns = [
    ...

    # add the following line
    url(r'^ssearch/(?P<fname>[\w-]+)/$', contacts.views.SimpleSearchListContactView.as_view(), name='simplesearchlistcontactview', ),
    # N.B. Note the use of the Regex Named Group, <fname>, sent to the SimpleSearchListContactView as a (name, value) pair used by self.kwargs.
    ...
]

Test 1:
Navigate to http://127.0.0.1:8000/contacts/ssearch/J/
Expected Result
Contact List is displayed showing:

Contacts
    Joe Codeswell j@j.com
    Jayne Mansfield j@thebomb.com

Test 2:
Navigate to http://127.0.0.1:8000/contacts/ssearch/a/
Expected Result
Contact List is displayed showing:

Contacts
    Jayne Mansfield j@thebomb.com

Test 3:
Navigate to http://127.0.0.1:8000/contacts/ssearch/o/
Expected Result
Contact List is displayed showing:

Contacts
    Joe Codeswell j@j.com

4.13.2 Parametric Search – ListContactView

Let’s make & test our – ParametricSearchListContact — View, Template & URL Pattern.
View: Edit CODE_HOME/mysite/contacts/views.py adding code after our SimpleSearchListContactView class definition.

...
class SimpleSearchListContactView(ListView):
   ...
# added code under here
class ParametricSearchListContactView(ListView):
    model = Contact
    template_name = 'contacts/contact_list.html'

    def get_queryset(self):
        qs = super(ParametricSearchListContactView, self).get_queryset()
        field_lookups_dict = {
            '%s__%s'%(self.kwargs['field_name'], self.kwargs['match_cmd']):self.kwargs['value'],
            # for a def of each match_cmd see Field lookups - https://docs.djangoproject.com/en/1.9/ref/models/querysets/#field-lookups
        }
        return qs.filter(**field_lookups_dict)

Template: We reuse the Template from the ListContactView, contacts/contact_list.html. Nothing else to do here.
URL Pattern: Add our Parametric Search ListContactView Url Pattern to CODE_HOME/mysite/contacts/urls.py as follows.

urlpatterns = [
    ...

    # add the following line
    url(r'^psearch/(?P<field_name>[\w-]+)/(?P<match_cmd>[\w-]+)/(?P<value>[\w-]+)/$', contacts.views.ParametricSearchListContactView.as_view(), name='parametricsearchlistcontactview', ),

    ...
]

Test 1:
Navigate to http://127.0.0.1:8000/contacts/psearch/first_name/contains/J/
Expected Result
Contact List is displayed showing:

Contacts
    Joe Codeswell j@j.com
    Jayne Mansfield j@thebomb.com

Test 2:
Navigate to http://127.0.0.1:8000/contacts/psearch/email/contains/bomb/
Expected Result
Contact List is displayed showing:

Contacts
    Jayne Mansfield j@thebomb.com

Test 3:
Navigate to http://127.0.0.1:8000/contacts/psearch/last_name/endswith/eld/
Expected Result
Contact List is displayed showing:

Contacts
    Jayne Mansfield j@thebomb.com

Test 4:
Navigate to http://127.0.0.1:8000/contacts/psearch/last_name/exact/Codeswell/
Expected Result
Contact List is displayed showing:

Contacts
    Joe Codeswell j@j.com

Test 5:
Navigate to http://127.0.0.1:8000/contacts/psearch/last_name/startswith/Man/
Expected Result
Contact List is displayed showing:

Contacts
    Jayne Mansfield j@thebomb.com

Test 6:
Navigate to http://127.0.0.1:8000/contacts/psearch/email/regex/bo/
Expected Result
Contact List is displayed showing:

Contacts
    Jayne Mansfield j@thebomb.com

Test 7:
Navigate to http://127.0.0.1:8000/contacts/psearch/last_name/regex/ode/
Expected Result
Contact List is displayed showing:

Contacts
    Joe Codeswell j@j.com

In summary, see the Django Docs “Field lookups” for a full list of match_cmds like
contains
endswith
exact
startswith
reges
– etc.

4.13.3 Searches (Both Simple & Parametric) for CRUD List View

The CRUD ListView Simple & Parametric Searches are PRETTY MUCH THE SAME AS those for the NON-CRUD ListView.
There are ONLY 2 differences:

  1. The template_name = ‘contacts/crudlist.html’ NOT ‘contacts/contact_list.html’.
  2. The class names – these occur in 2 places in each view.

Let’s make ** & **test BOTH our – SimpleSearchCRUDListContactView & ParametricSearchCRUDListContactView — Views, Templates & URL Patterns AT THE SAME TIME.
View: Edit CODE_HOME/mysite/contacts/views.py adding code for BOTH after our ParametricSearchListContactView class definition.

...
class ParametricSearchListContactView(ListView):
   ...
# added code under here (2 classes for Simple & Parametric Search CRUD ListViews)
class SimpleSearchCRUDListContactView(ListView):

    model = Contact
    template_name = 'contacts/crudlist.html'    

    def get_queryset(self):
        qs = super(SimpleSearchCRUDListContactView, self).get_queryset()

        return qs.filter(first_name__contains=self.kwargs['fname'])
        # "first_name" is the first_name field in our Contacts model
        # "contains" is one of Django's "Field lookups"
        # "'fname'" is passed in from our url <fname> Regex Named Group  


class ParametricSearchCRUDListContactView(ListView):
    model = Contact
    template_name = 'contacts/crudlist.html'


    def get_queryset(self):
        qs = super(ParametricSearchCRUDListContactView, self).get_queryset()
        field_lookups_dict = {
            '%s__%s'%(self.kwargs['field_name'], self.kwargs['match_cmd']):self.kwargs['value'],
            # for a def of each match_cmd see Field lookups - https://docs.djangoproject.com/en/1.9/ref/models/querysets/#field-lookups
        }

        return qs.filter(**field_lookups_dict)

N.B. These classes are the same as the previous 2 EXCEPT:

  1. template_name = ‘contacts/crudlist.html’ NOT ‘contacts/contact_list.html’
  2. Class names are new:
    • Lines 5 & 10 “SimpleSearchCRUDListContactView”
    • Lines 18 & 23 “ParametricSearchCRUDListContactView”

Template: We reuse the Template from the CRUDListView, contacts/crudlist.html. Nothing else to do here.
URL Patterns: Add BOTH our Simple & Parametric Search CRUDListView Url Patterns to CODE_HOME/mysite/contacts/urls.py as follows.

urlpatterns = [
    ...

    # add the following 2 lines
    url(r'^ssearchc/(?P<fname>[\w-]+)/$', contacts.views.SimpleSearchCRUDListContactView.as_view(template_name='contacts/crudlist.htm'), name='ssearchc', ),
    url(r'^psearchc/(?P<field_name>[\w-]+)/(?P<match_cmd>[\w-]+)/(?P<value>[\w-]+)/$', contacts.views.ParametricSearchCRUDListContactView.as_view(template_name='contacts/crudlist.htm'), name='psearchc', ),

    ...
]

Test our CRUD Searches
Perform the “same”

  • 3 tests we did for Simple Searches
  • 7 tests we did for Parametrics Searches
    EXCEPT

  • Navigate to …/contacts/ssearchc/,,, NOT …/contacts/ssearch/… AND

  • Navigate to …/contacts/psearchc/,,, NOT …/contacts/psearch/…

EXPECTED RESULTS
Same as before
EXCEPT items are listed with read, update & delete links.

Content

Click here to go back to Abstract (i.e. Top of Document)

1. Prerequisites
– 1.1 Brief basics of urls and Views
2. Create a Project i.e. a Site
– 2.1 Run django-admin startproject
– 2.2 Verify the Site
3. Hello App
– 3.1 Run manage.py startapp
– 3.2 Add hello to settings.py
– 3.3 Add an App url
– 3.4 Create our App’s urls.py file
– 3.5 Create a View
– 3.6 Test our View
4. Contacts App
– 4.1 Run manage.py startapp contacts
– 4.2 Add contacts to settings.py like for hello
– 4.3 Add an App url like for hello
– 4.4 Create our App’s urls.py file
– 4.5 Create a View
– 4.6 Test our View
– 4.7 Add a Model
– 4.8 Run makemigrations
– 4.9 Run migrate
– 4.10 Load the db using Admin
—- 4.10.1 Register Model for Admin
—- 4.10.2 Load the db
– 4.11 List our existing contacts
—- 4.11.1 Create a ListView
—- 4.11.2 Create a ListView Template
—— 4.11.2.1 Define Site Template Locations
—— 4.11.2.2 Define our ListView Template
—- 4.11.3 Hook Up our ListView URL
—- 4.11.4 Test our ListView URL
– 4.12 CRUD Contacts
—- 4.12.1 Make a CRUD List View
—- 4.12.2 Make a CRUD List Template
—- 4.12.3 Hook up the CRUD List URL
—- 4.12.4 Test CRUD List View
—- 4.12.5 Make CRUD Create Contact
—- 4.12.6 Make CRUD Read Contact
—- 4.12.7 Make CRUD Update Contact
—- 4.12.8 Make CRUD Delete Contact
– 4.13 Search Contacts
—- 4.13.1 Simple Search – ListContactView
—- 4.13.2 Parametric Search – ListContactView
—- 4.13.3 Searches (Both Simple & Parametric) for CRUD List View

Execute Shell Commands from a Python GUI App

GUI Execute Commands

Here’s my Python – TKinter App that takes a set of Shell Commands from a .json file and displays them for individual execution when a button is clicked. Write these frequently used commands once and let the GUI program remember their syntax.

Keep 3 files in the same directory:

  • guiExecuteCommands.py The TKInter Program
  • config.json The Configuration Data
  • cmds.json The Initial Command Set

Contents

  • The TKInter Program File – guiExecuteCommands.py
  • The Configuration Data File – config.json
  • The Initial Command Set – cmds.json
  • A Screenshot of the TKInter Program
  • A run log of the program

The TKInter Program File – guiExecuteCommands.py

Here is the guiExecuteCommands.py file.

#!/usr/bin/python
# -*- coding: utf-8 -*-
# guiExecuteCommands.py
import sys, os, time, subprocess, json, atexit

# if not('.' in sys.path): sys.path.append('.')
# from MyCommands import cmds

import Tkinter, tkFileDialog

class Theapp(Tkinter.Tk):
    def __init__(self,parent):

        # joe's init
        with open('config.json') as data_file:
            cfg_data = json.load(data_file)
        self.cfg_data = cfg_data 

        with open(cfg_data[0]['commandsf']) as data_file:
            cmds = json.load(data_file)
        self.cmds = cmds 


        atexit.register(self.on_exit) # http://stackoverflow.com/a/28664350/601770


        # /joe's init




        Tkinter.Tk.__init__(self,parent)
        self.parent = parent
        self.initialize()

    def on_exit(self):
        ''' http://stackoverflow.com/a/28664350/601770  
            save the state that has been changed
        '''

        self.cfg_data[0]['workingd'] = self.cwd.get()


        print "self.cwd.get(): %s"%(self.cwd.get())
        print "self.cfg_data: %s"%(self.cfg_data)

        with open('config.json', 'w') as f:
            json.dump(self.cfg_data, f)

    def execute_cmd(self, rnum):
        ''' C:\1d\PythonPjs\subprocessPjs\executeCommandsPj\executeCommands\executeCommands.py

        '''
        commandstring = self.cmd_entries[rnum].get()
        cwdpath = self.cwd.get()
        outstr = subprocess.check_call(commandstring, cwd=cwdpath, shell=True)  #  Convenience Function
        print outstr


    ## http://stackoverflow.com/a/16777523/601770   
    def bclick(self, rnum):
        def click():            
            self.execute_cmd(rnum)
        return click



    def browse2cwd(self):   
        """
        http://stackoverflow.com/questions/11295917/how-to-select-a-directory-and-store-the-location-using-tkinter-in-python
        http://tkinter.unpythonic.net/wiki/tkFileDialog
            find "def askdirectory(self):"
        """   
        tkFileDiaOpt4Dir = {}
        tkFileDiaOpt4Dir['initialdir'] = self.cwd.get()
        tkFileDiaOpt4Dir['mustexist'] = False
        tkFileDiaOpt4Dir['parent'] = self
        tkFileDiaOpt4Dir['title'] = 'Browse to set CWD.'

        self.cwd.set( tkFileDialog.askdirectory(**tkFileDiaOpt4Dir) )

        print "browse2cwd:  self.cwd.get(): %s"%(self.cwd.get())

    def initialize(self):
        self.cwd = Tkinter.StringVar()
        self.cwd.set(self.cfg_data[0]['workingd'])

        self.grid()  # make this be a grid

        ### add the gui elements in here

        ## browse to working directory
        label = Tkinter.Label(self, text="Working Directory", fg="white",bg="blue")
        label.grid(column=0,row=0,columnspan=3,sticky='W')

        self.cwd_entry = Tkinter.Entry(self, width=80, textvariable=self.cwd)
        self.cwd_entry.grid(column=0,row=1,columnspan=2, sticky='W') # http://effbot.org/tkinterbook/entry.htm

        b = Tkinter.Button(self,text=u"Browse", command=self.browse2cwd)
        b.grid(column=2,row=1) 

        toprowscount = 2


        ## command row elements len == len(MyCommands.cmds)
        self.buttons = []
        self.cmd_entries = []
        for rnum, cmd in enumerate(self.cmds):


            label = Tkinter.Label(self, anchor="w",text=cmd['label'], fg="white",bg="blue")
            label.grid(column=0,row=rnum+toprowscount,sticky='EW')

            self.cmd_entries.append( Tkinter.Entry(self, width=60) )
            self.cmd_entries[-1].grid(column=1,row=rnum+toprowscount,sticky='EW'); self.cmd_entries[-1].insert(0, (cmd['cmd']+' '+ cmd['args']))


            b = Tkinter.Button(self,text=u"Ex%s"%(rnum), command=self.bclick(rnum))
            b.grid(column=2,row=rnum+toprowscount) 

            self.buttons.append(b)


        ## Finish up
        self.grid_columnconfigure(0,weight=1) # make resizable
        self.resizable(True,False)            # make resizable ONLY HORIZONTALLY




if __name__ == "__main__":
    app = Theapp(None)
    app.title('Gui Execute Comands')
    app.mainloop()

The Configuration Data File – config.json

Here is the config.json file.

[{
   "programd": "C:\\1d\\PythonPjs\\GUIpjs\\guiExecuteCommandPj\\guiExecuteCommand", 
   "commandsf": "C:\\1d\\PythonPjs\\GUIpjs\\guiExecuteCommandPj\\guiExecuteCommand\\cmds.json",  
   "configd": "C:\\1d\\PythonPjs\\GUIpjs\\guiExecuteCommandPj\\guiExecuteCommand", 
   "workingd": "C:/1d/CrossPlatformPjs/CordovaPjs/cordovaAngular00Pj/cordovaAngular00"
}]

The Initial Command Set – cmds.json

Here is the (INITIAL) cmds.json file

[
  {"cmd": "cordova", "args": "run android", "label": "android build 4 device"}, 
  {"cmd": "cordova", "args": "emulate android", "label": "android build 4 emulation"}, 
  {"cmd": "dir", "args": "*", "label": "dir of working dir"}
]  

A Screenshot of the TKInter Program

Here is a screenshot of the program.

guiExecuteShot

A run log of the program

Here is a run log.

Note: if the user changed the working directory, on exit the new working directory will be saved in the config.json file.

C:\1bat\guiExecuteCommands>guiExecuteCommands.py
browse2cwd:  self.cwd.get(): C:/1d/CrossPlatformPjs/Building Single Page Web Apps with AngularJS PJ
 Volume in drive C is Windows
 Volume Serial Number is F2FB-4810

 Directory of C:\1d\CrossPlatformPjs\Building Single Page Web Apps with AngularJS PJ

12/23/2015  09:58 AM    <DIR>          .
12/23/2015  09:58 AM    <DIR>          ..
12/23/2015  11:11 AM    <DIR>          zetc
               0 File(s)              0 bytes
               3 Dir(s)  900,352,217,088 bytes free
0
self.cwd.get(): C:/1d/CrossPlatformPjs/Building Single Page Web Apps with AngularJS PJ
self.cfg_data: [{u'programd': u'C:\\1d\\PythonPjs\\GUIpjs\\guiExecuteCommandPj\\guiExecuteCommand', u'commandsf': u'C:\\1d\\PythonPjs\\GUIpjs\\guiExecuteCommandPj\\guiExecuteCommand\\cmds.json', u'configd': u'C:\\1d\\PythonPjs\\GUIpjs\\guiExecuteCommandPj\\guiExecuteCommand', u'workingd': u'C:/1d/CrossPlatformPjs/Building Single Page Web Apps with AngularJS PJ'}]

C:\1bat\guiExecuteCommands>

Analyze Python cProfile stats created with pstats.dump_stats() off line.

Background

In order to improve Code Performance, I need to find functions that are good candidates for Cython implementation.

Here’s the link to the Python Docs for Code Profiling. Pay attention to cProfile.

I asked and answered this on stack overflow here.

Preamble

Previously I have essentially done the following:

import cProfile, pstats, StringIO
pr = cProfile.Profile()
pr.enable()
# ... my code did something ...
pr.disable()
s = StringIO.StringIO()
sortby = 'cumulative'
ps = pstats.Stats(pr, stream=s).sort_stats(sortby)

ps.dump_stats('stats.dmp')  # dump the stats to a file named stats.dmp

So now I have the file named ‘stats.dmp’ stored off line.

Here’s how I used pstats to analyze the ‘stats.dmp’ file for human consumption off line.

The Code

#!/usr/local/bin/python2.7
# -*- coding: UTF-8 -*-
"""analyze_dmp.py takes the file INFILEPATH [a pstats dump file] Producing OUTFILEPATH [a human readable python profile]
Usage:   analyze_dmp.py INFILEPATH  OUTFILEPATH
Example: analyze_dmp.py stats.dmp   stats.log
"""
import sys, os
import cProfile, pstats, StringIO

def analyze_dmp(myinfilepath='stats.dmp', myoutfilepath='stats.log'):
    out_stream = open(myoutfilepath, 'w')
    ps = pstats.Stats(myinfilepath, stream=out_stream)
    sortby = 'cumulative'

    ps.strip_dirs().sort_stats(sortby).print_stats(.3)  # plink around with this to get the results you need

NUM_ARGS = 2
def main():
    args = sys.argv[1:]
    if len(args) != NUM_ARGS or "-h" in args or "--help" in args:
        print __doc__
        s = raw_input('hit return to quit')
        sys.exit(2)
    analyze_dmp(myinfilepath=args[0], myoutfilepath=args[1])

if __name__ == '__main__':
    main()

I tested this with a .dmp file created on Linux & analyzed on Windows XP. It worked FINE. The above Python file is named, “analyze_dmp.py”.

An Additional Resource

I also found out how to use the Python command line to analyze stats.dmp in an interactive way here.

Here’s the command line i used on Windows XP.

ipython -m pstats stats.dmp