Building Skills in Python

A Programmer’s Introduction to Python

Language Basics

The Processing View

A programming language involves two closely interleaved topics. On one hand, there are the procedural constructs that process information inside the computer, with visible effects on the various external devices. On the other hand are the various types of data structures and relationships for organizing the information manipulated by the program.

This part describes the most commonly-used Python statements, sticking with basic numeric data types. Data Structures will present a reasonably complete set of built-in data types and features for Python. While the two are tightly interwoven, we pick the statements as more fundamental because we can (and will) add new data types. Indeed, the essential thrust of object-oriented programming (covered in Data + Processing = Objects) is the creation of new data types.

Some of the examples in this part refer to the rules of various common casino games. Knowledge of casino gambling is not essential to understanding the language or this part of the book. We don’t endorse casino gambling. Indeed, many of the exercises reveal the magnitude of the house edge in most casino games. However, casino games have just the right level of algorithmic complexity to make for excellent programming exercises.

We’ll provide a little background on Python in Background and History. From there, we’ll move on to installing Python in Python Installation.

In Simple Numeric Expressions and Output we’ll introduce the print statement (and print() function); we’ll use this to see the results of arithmetic expressions including the numeric data types, operators, conversions, and some built-in functions. We’ll expand on this in Advanced Expressions.

We’ll introduce variables, the assignment statement, and input in Variables, Assignment and Input, allowing us to create simple input-process-output programs. When we add truth, comparisons, conditional processing in Truth, Comparison and Conditional Processing, and iteration in Loops and Iterative Processing, we’ll have all the tools necessary for programming. In Functions and Additional Notes On Functions, we’ll show how to define and use functions, the first of many tools for organizing programs to make them understandable.

Data Structures

The Data View

Computer programs are built with two essential features: data and processing. We started with processing elements of Python. We’re about to start looking at data structures.

In Language Basics, we introduced almost all of the procedural elements of the Python language. We started with expressions, looking at the various operators and data types available. We described fourteen of the approximately 24 statements that make up the Python language.

  • Expression Statement. For example, a function evaluation where there is no return value. Examples include the print() function.
  • import. Used to include a module into another module or program.
  • print. Used to provide visible output. This is being replaced by the print() function.
  • assignment. This includes the simple and augmented assignment statements. This is how you create variables.
  • del. Used (rarely) to remove a variable, function, module or other object.
  • if. Used to conditionally perform suites of statements. This includes elif and else statements.
  • pass. This does nothing, but is a necessary syntactic placeholder for an if or while suite that is empty.
  • assert. Used to confirm the program is in the expected state.
  • for and while. Perform suites of statements using a sequence of values or while a condition is held true.
  • break and continue. Helpful statements for short-cutting loop execution.
  • def. Used to define a new function.
  • return. Used to exit a function. Provides the return value from the function.
  • global. Used adjust the scoping rules, allowing local access to global names. We discourage its use in The global Statement.

The Other Side of the Coin. The next chapters focus on adding various data types to the basic Python language. The subject of data representation and data structures is possibly the most profound part of computer programming. Most of the killer applications – email, the world wide web, relational databases – are basically programs to create, read and transmit complex data structures.

We will make extensive use of the object classes that are built-in to Python. This experience will help us design our own object classes in Data + Processing = Objects.

We’ll work our way through the following data structures.

  • Sequences. In Sequences: Strings, Tuples and Lists we’ll extend our knowledge of data types to include an overview various kinds of sequences: strings, tuples and lists. Sequences are collections of objects accessed by their numeric position within the collection.

    • In Strings we describe the string subclass of sequence. The exercises include some challenging string manipulations.
    • We describe fixed-length sequences, called tuple s in Tuples.
    • In Lists we describe the variable-length sequence, called a list. This list sequence is one of the powerful features that sets Python apart from other programming languages. The exercises at the end of the list section include both simple and relatively sophisticated problems.
  • Mappings. In Mappings and Dictionaries we describe mappings and dictionary objects, called dict. We’ll show how dictionaries are part of some advanced techniques for handling arguments to functions. Mappings are collections of value objects that are accessed by key objects.

  • Sets. We’ll cover set objects in Sets. Sets are simple collections of unique objects with no additional kind of access.

  • Exceptions. We’ll cover exception objects in Exceptions. We’ll also show the exception handling statements, including try, except, finally and raise statements. Exceptions are both simple data objects and events that control the execution of our programs.

  • Iterables. The yield statement is a variation on return that simplifies certain kinds of generator algorithms that process or create create iterable data structures. We can iterate through almost any kind of data collection. We can also define our own unique or specialized iterations. We’ll cover this in Iterators and Generators.

  • Files. The subject of files is so vast, that we’ll introduce file objects in Files. The with statement is particularly helpful when working with files.

    Files are so centrally important that we’ll return files in Components, Modules and Packages. We’ll look at several of the file-related modules in File Handling Modules as well as File Formats: CSV, Tab, XML, Logs and Others..

In Functional Programming with Collections we describe more advanced sequence techniques, including multi-dimensional processing, additional sequence-processing functions, and sorting.

Deferred Topics. There are a few topics that need to be deferred until later.

Data + Processing = Objects

Encapsulating Data and Processing into Class Definitions

In Language Basics, we examined the core statements in the Python language. In Data Structures, we examined the built-in data structures available to us as programmers. Using these data structures gave us some hands-on experience with a number of classes. After using this variety of built-in objects, we are better prepared to design our own objects.

Classes introduces basics of class definitions and Advanced Class Definition introduces simple inheritance. We extend this discussion further to include several common design patterns that use polymorphism. In Some Design Patterns we cover a few common design patterns. Creating or Extending Data Types describes the mechanism for adding types to Python that behave like the built-in types.

We will spend some time on Python’s flexible notion of “attribute” in Attributes, Properties and Descriptors. It turns out that an attribute can be a simple instance variable or it can be a method function that manages an instance variable.

We’ll look at Python’s decorators in Decorators; this is handy syntax for assuring that specific aspects of a family of classes are implemented consistently. We’ll look at the various ways to define properties in objects.properties. Additionally, we’ll look how we can manage more sophisticated object protocols in Managing Contexts: the with Statement.

Data Types. We’ve looked at most of these data types in Data Structures. This is a kind of road-map of some of the most important built-in features.

  • None. A unique constant, handy as a placeholder when no other value is appropriate. A number of built-in functions return values of None to indicate that no useful work can be done.

  • NotImplemented. A unique constant, returned by special methods to indicate that the method is not implemented. See Numeric Type Special Methods for more information.

  • Numeric types have relatively simple values. These are immutable objects: they cannot have their values changed, but they can participate in numerous arithmetic and comparison operations like +, -, *, /, //, **.

    • Boolean. (bool) A variety of values are treated as logically false: False, 0, None, "", (), [], {}, set(). All other values are logically True.

    • Integer. (int) Typically 32-bit numbers with a range of -2,147,483,648 through 2,147,483,647.

    • Long. (long) These are specially coded integers of arbitrary length. They grow as needed to accurately represent numeric results. Literals end with L.

      In Python 3, the integer and long types will be unified and the remaining distinctions removed.

    • Float. (float) These are floating point, scientific notation numbers. They are represented using the platform’s floating point notation, so ranges and precisions vary. Typically these are called “double precision” in other languages, and are often 64-bits long.

    • Complex. (complex) These are a pair of floating point numbers of the form a + bj, where a is the real part and b is the “imaginary” part. j = \sqrt{-1}.

  • Sequence. Collections of objects identified by their order or position.

    • Immutable sequences are created as needed and can be used but never changed.

      • String. (str) A sequence of individual ASCII characters.

      • Unicode. (unicode) A sequence of individual Unicode characters.

        In Python 3, String and Unicode will be unified into a single class named str that includes features of unicode.

      • Tuple. (tuple) A sequence of a fixed number of Python items. Literals look like ( expression, ... 〉 )

    • Mutable sequences can be created, appended-to, changed, and have elements deleted.

      • List. (list) A sequence Python items. Literals look like [ expression, ... 〉 ] Operations like append(), pop() and sort() can be used to change lists.
  • Set and Frozenset. (set, frozenset) Collections of objects. The collection is neither ordered nor keyed. Each item stands for itself. A set is mutable; we can append, change and delete elements from a set. A frozenset is immutable.

  • Mapping. Collections of objects identified by keys instead of order.

    • Dictionary. (dict) A collection of objects which are indexed by other objects. It is like a sequence of key:value pairs, where keys can be found efficiently. Any Python object can be used as the value. Keys have a small restriction: mutable lists and other mappings cannot be used as keys. Literals look like { key : value, ... 〉 }
  • File. (classname:file) Python supports several operations on files, most notably reading, writing and closing. Python also provides numerous modules for interacting with the operating system’s management of files.

  • Callable. When we create a function with the def statement, we create a callable object. We can also define our own classes with a special method of __call__() to make a callable object that behaves like a function.

  • Class. What we’ll cover in this part.

There are numerous additional data structures that are part of Python’s implementation internals; they are beyond the scope of this book.

One of the most powerful and useful features of Python, is its ability to define new classes. The next chapters will introduce the class and the basics of object-oriented programming.

Components, Modules and Packages

Organization and Deployment

The basic Python language is rich with features. These include several sophisticated built-in data types (Data Structures), numerous basic statments (Language Basics), a variety of common arithmetic operators and a library of built-in functions. In order to keep the basic Python kernel small, relatively feature features are built-in. A small kernel means that Python interpreters can be provided in a variety of software application, extending functionality of the application without bloating due to a large and complex command language.

The more powerful and sophisticated features of Python are separated into extension modules. There are several advantages to this. First, it allows each program to load only the relevant modules, speeding start-up. Second, it allows additional modules to be added easily. Third, it allows a module to be replaced, allowing you to choose among competing solutions to a problem.

The second point above, easily adding modules, is something that needs to be emphasized. In the Python community, this is called the batteries included principle. The ideal is to make Python directly applicable to just about any practical problem you may have.

Some modules have already been covered in other chapters. In The math Module we covered math and random modules. In Strings we covered the string module.

Overview of this part. This part will cover selected features of a few modules. The objective is to introduce some of the power of key Python modules and show how the modules are used to support software development. This isn’t a reference, or even a complete guide to these modules. The standard Python Library documentation and other books describe all available modules in detail. Remember that Python is an open-source project: in some cases, you’ll have to read the module’s source to see what it really does and how it works.

This part provides a general overview of how to create Python modules in Modules. We’ll distinguish package and module in Packages.

We’ll overview the Python Library in The Python Library.

Module Details. We cover several essential modules in some detail.

Programs – The Ultimate Modules. In a sense a top-level program is a module that does something useful. It’s important understand “programs” as being reusable modules. Eventually most really useful programs get rewritten and merged into larger, more sophisticated programs.

In Programs: Standing Alone this part covers modules essential for creating polished, complete stand-alone programs. This includes the getopt and optparse modules.

The final chapter, Architecture: Clients, Servers, the Internet and the World Wide Web covers integration among programs using the client-server programming model. This includes a number of modules that are essential for creating networked programs.

Projects

Projects to Build Skills

Programming language skills begin with the basic syntax and semantics of the language. They advance through the solution of small exercises and are refined through solving more complete problems.

“Real-world” applications, used every day in business and research, are less than ideal for learning a programming language. The business-oriented problems often have a very narrow in focus; the solutions are dictated by odd budgetary constraints or departmental politics. Reasearch problems are also narrowly focused, often lacking a final “application” to surround the interesting parts of the programming and create a final, finished product.

This part provides several large exercises that provide for more advanced programming than the smaller exercises at the end of each section. These aren’t real-world in scope, but they are quite a bit larger than the small exercises at the end of each chapter.

These are ranked in order of difficulty.

Back Matter

Indices and Tables

Production Notes

The following toolset was used for production of this book.

  • Python 2.6.3.
  • Sphinx 0.6.3.
  • Docutils 0.5.
  • Komodo Edit 5.2.2.
  • pyPDF 1.12.
  • MacTeX-2008.

Table Of Contents

Next topic

Preface

This Page