You’ll need to read this book when you have the following three things happening at the same time:
You’ll want to read this book if you are tinkerer who likes to know how things really work. For many people, a computer is just an appliance. You may not find this satisfactory, and you want to know more. People who tinker with computers are called hackers, and you are about to join their ranks.
Python is what you’ve been looking for. It is an easy-to-use tool that can do any kind of processing on any kind of data. Seriously: any processing, any data. Programming is the term for setting up a computer to do the processing you define on your data. Once you learn the Python language, you can solve your data processing problem.
Our objective is to get you, a non-programming newbie, up and running. When you’re done with this book, you’ll be ready to move on to a more advanced Python book. For example, a book about the Python libraries. You can use these libraries can help you build high-quality software with a minimum of work.
This book is about many things. The important topics include Python, programming, languages, data, processing, and some of the skills that make up the craft of programming. We’ll talk about the core intellectual tools of abstraction, algorithms and the formality of computer languages. We’ll also touch on math and logic, statistics, and casino games.
Python. Python is a powerful, flexible toolbox and workbench that can help solve your data processing problem. If you need to write customized software that does precisely what you want, and you want that software to be readable, maintainable, adaptable, inexpensive and make best use of your computer, you need Python.
Here’s a very important distinction:
What does this distinction mean? First, there is an opportunity for us to confuse Python (the program) and Python (the language). We’ll attempt to be as clear as we can on the things the Python program does when you give it commands in the Python Language. For people very new to computers, this raises questions like “what is a programming language?” and “why can’t it just use English?” and “what if I’m not good with languages?” We’ll return to these topics in Concepts FAQ’s. For now, we’ll emphasize the point that the Python language is more precise than English, but still very easy to read and write.
The other thing that the distinction between program and language means is that we will focus our efforts on learning the language. The data processing we want to perform will be completely defined by a sequence of statements in the Python language. Learning a computer language isn’t a lot different from learning a human language, making our job relatively easy. We’ll be reading and writing Python in no time.
Programming. When we’ve written a sequence of Python statements, we can then use that sequence over and over again. We can process different sets of data in a standard, automatic fashion. We’ve created a program that can automate data processing tasks, replacing tedious or error-prone pointing and clicking in other software tools. Also, we can create programs that do things that other desktop tools can’t do at all.
The big picture is this: the combination of the Python program plus a unique sequence of Python language statements that we create can have the effect of creating a new application for our computer. This means that our new application uses the existing Python program as its foundation. The Python program, in turn, depends on many other libraries and programs on your computer. The whole structure forms a kind of technology stack, with your program on top, controlling the whole assembly.
Languages. We’ll look at three facets of a programming language: how you write it, what it means, and the additional practical considerations that make a program useful. We’ll use these three concepts to organize our presentation of the language. We need to separate these concepts to assure that there isn’t a lot of confusion between the real meaning and the ways we express that meaning.
The sentences “Xander wrote a tone poem for chamber orchestra” and “The chamber orchestra’s tone poem was written by Xander” have the same meaning, but express it different ways. They have the same semantics, but different syntax. For example, in one sentence the verb is “wrote” , in the other sentence it is “was written by” : different forms of the verb to write. The first form is written in active voice, and second form is called the passive voice. Pragmatically, the first form is slightly clearer and more easily understood.
The syntax of the Python language is covered here, and in the Python Reference Manual [PythonRef]. Python syntax is simple, and very much like English. We’ll provide many examples of language syntax. We’ll also provide additional tips and hints focused on the newbies and non-programmers. Also, when you install Python, you will also install a Python Tutorial [PythonTut] that presents some aspects of the language, so you’ll have at least three places to learn syntax.
The semantics of the language specify what a statement really means. We’ll define the semantics of each statement by showing what it makes the Python program do to your data. We’ll also be able to show where there are alternative syntax choices that have the same meaning. In addition to semantics being covered in this book, you’ll be able to read about the meaning of Python statements in the Python Reference Manual [PythonRef], the Python Tutorial [PythonTut], and chapter two of the Python Library Reference [PythonLib].
In this book, we’ll try to provide you with plenty of practical advice. In addition to breaking the topic into bite-sized pieces, we’ll also present lots of patterns for using Python that you can apply to real-world problems.
Extensions. Part of the Python technology stack are the extension libraries. These libraries are added onto Python, which has the advantage of keeping the language trim and fit. Software components that you might need for specialized processing are kept separate from the core language. Plus, you can safely ignore the components you don’t need.
This means that we actually have two things to learn. First, we’ll learn the language. After that, we’ll look at a few of the essential libraries. Once we’ve seen that, we can see how to make our own libraries, and our own application programs.
Programming and Computer Skills. We’re going to focus on programming skills, which means we have to presume that you already have general computer skills. You should fit into one of these populations.
What skills will you need? How will we build up your new skills?
Skills You’ll Need. This book assumes an introductory level of skill with any of the commonly-available computer systems. Python runs on almost any computer; because of this, we call it platform-independent. We won’t presume a specific computer or operating system. Some basic skills will be required. If these are a problem, you’ll need to brush up on these before going too far in this book.
How We Help. Newbie programmers with an interest in Python are our primary audience. We provide specific help for you in a number of ways.
When you’ve finished with this book you should be able to do the following.
A Note on Clue Absorption. Learning a programming language involves accumulating many new and closely intertwined concepts. In our experience teaching, coaching and doing programming, there is an upper limit on the “Clue Absorption Rate” . In order to keep below this limit, we’ve found that it helps to build up the language as ever-expanding layers. We’ll start with a very tiny, easy to understand subset of statements; to this we’ll add concepts until we’ve covered the entire Python language and all of the built-in data types.
Our part of the agreement is to do things in small steps. Here’s your part: you learn a language by using it. In order for each layer to act as a foundation for the following layers, you have to let it solidify by doing small programming exercises that exemplify the layer’s concepts. Learning Python is no different from learning Swedish. You can read about Sweden and Swedish, but you must actually use the language to get it off the page and into your head. We’ve found that doing a number of exercises is the only way to internalize each language concept. There is no substitute for hands-on use of Python. You’ll need to follow the examples and do the exercises. As you can probably tell from this paragraph, we can’t emphasize this enough.
The big difference between learning Python and learning Swedish is that you can immediately interact with the Python program, doing real work in the Python language. Interacting in Swedish can more difficult. The point of learning Swedish is to interact with people: for example, buying some kanelbulle (cinnamon buns) for fika (snack). However, unless you live in Sweden, or have friends or neighbors who speak Swedish, this interactive part of learning a human language is difficult. Interacting with Python only requires a working computer, not a trip to Kiruna.
Also, your Swedish phrase-book gives you little useful guidance on how to pronounce words like sked (spoon) or sju (seven); words which are notoriously tricky for English-speakers like me. Further, there are some regional accents within Sweden, making it more difficult to learn. Python, however, is a purely written language so you don’t have subtleties of pronunciation, you only have spelling and grammar.
This book falls into fourteen distinct parts. To manage the clue absorption rate, the parts are organized in a way that builds up the language in layers from simple, central concepts to more advanced features. Each part introduces a few new concepts. Programming exercises are provided to encourage further exploration of each layer.
Some programming languages (like Pascal or Basic) were specifically designed to help teach programming. Most other programming languages (like Python) are designed for doing the practical work of solving information processing problems. One consequence of this is that Python is a tightly integrated whole. Some features of the language will have both simple and advanced semantics. In many cases some simple-looking features will actually depend on some more advanced parts of the language. This forces us to revisit some subjects several times, first for an introduction, then for more in-depth treatment.
Chickens and Eggs. One subtext woven into this book is the two-sided coin labeled “data processing” . The processing side of the coin reflects the imperative-voice verb statements in the Python language. This active sense of “first do this, then do that” is central to programming. On the other side of the coin, we have the data side, which includes numbers, strings of letters, related groups of values, lists of values and relationships between values. Often, when we think of computer data, we think of files. The way we structure our data is also central to programming.
Since they’re both central, and hopelessly intertwined, Data and Processing have a chicken-and-egg relationship. We could cover either of these topics first and get to the other second. In this book, we had to choose and we elected to look at processing first, and then, in Getting Our Bearings, switching over to the data side.
The other topics that weave through this book are the design, debugging and testing skills you’ll need to grow. We’ll develop these skills through hands-on use, so each chapter has five kinds of information.
Some Big Problems. There are a couple of problems that we’ll use throughout this book to show how you use Python. Both problems are related to casino games. We don’t embrace gambling; indeed, as you work through these sample problems, you’ll see precisely how the casino games are rigged to take your money. We do, however, like casino games because they are moderately complex and not very geeky. Really complex problems require whole books just to discuss the problem and its solution. Simple problems can be solved with a spreadsheet. In the middle are problems that require Python.
We’ll provide some of the rules for Roulette in Two Minimally-Geeky Problems : Examples of Things Best Done by Customized Software as well as some of the rules for Craps. We’ll look at a couple of interesting casino gambling problems in this chapter that will give us a representative problem that we can solve with Python programming.
Getting Started. Getting Started introduces the basics of computers, languages and Python. About Computers defines the basic concepts we’ll be working with. About Programs will more fully define a program and the art of programming. Let There Be Python: Downloading and Installing covers installation of Python. Two Minimally-Geeky Problems : Examples of Things Best Done by Customized Software gives an overview of two problems we’ll use Python to solve. Why Python is So Cool provides some history and background on Python.
Using Python. Using Python introduces using Python and the IDLE development environment. We’ll cover direct use of Python in Instant Gratification : The Simplest Possible Conversation. We’ll cover IDLE in IDLE Time : Using Tools To Be More Productive.
Additional sections will add depth to this material as we explore more of the language. Turning Python Loose With a Script shows how to control Python with a script of statements. Turning Python Loose with More Sophisticated Scripts will make use of the Python control statements for more sophisticated scripts.
Processing. Arithmetic and Expressions introduces the basic features of the Python language. Simple Arithmetic : Numbers and Operators includes the basic arithmetic operations and numeric types. Better Arithmetic Through Functions introduces the most useful built-in functions. Special Ops : Binary Data and Operators covers some additional operators for more specialized purposes. Peeking Under the Hood has some additional topics that may help you get a better grip on how Python works.
Programming Essentials introduces the essential programming constructs for input, processing and output. Seeing Results : The print Statement shows how to do output with the print statement. Turning Python Loose With a Script shows how to control Python with a script of statements. Expressions, Constants and Variables introduces variables and the assignment statement. We’ll cover some additional assignment topics in Assignment Bonus Features, including multiple assignment and how to make best use of the Python shell. Can We Get Your Input? shows the two simple input functions.
Some Self-Control introduces the various ways to control which statements execute. Truth and Logic : Boolean Data and Operators adds truth and conditions to the language. We’ll look at comparisons in Making Decisions : The Comparison Operators. Processing Only When Necessary : The if Statement adds conditional and While We Have More To Do : The for and while Statements adds iterative processing statements. In Becoming More Controlling we’ll cover some additional topics in control. Turning Python Loose with More Sophisticated Scripts will make use of these control statements for more sophisticated scripts.
Organizing Programs with Function Definitions shows how to define functions to organize a program. Adding New Verbs : The def Statement introduces the basic function definition and use. From there we’ll look at Extra Functions: math and random. Flexibility and Clarity : Optional Parameters, Keyword Arguments adds some useful features to these basic. A Few More Function Definition Tools describes concepts like returning multiple values.
After introducing some basic types of collections in the next part, we’ll return to the language topics in Additional Processing Control Patterns. This will add exceptions in The Unexpected : The try and except statements and generators in Looping Back : Iterators, the for statement, and the yield statement.
Course Change. Programming is all about data and processing. Up to this point, we’ve focused on processing. From this point forward, we’ll focus on data. Since these are two sides of the same coin, there’s no absolute separation, it’s only a matter of focus. Getting Our Bearings will clarify this relationship between data and processing.
Data. We’ll start covering the data side of data processing in Basic Sequential Collections of Data, which is an overview of the sequential collections. Collecting Items in Sequence extends the data types to include various kinds of sequences. These include Sequences of Characters : str and Unicode, Doubles, Triples, Quadruples : The tuple and Flexible Sequences : the list. We’ll look at some additional topics in Common List Design Patterns.
We’ll revisit some processing elements in Additional Processing Control Patterns. This will include The Unexpected : The try and except statements as well as Looping Back : Iterators, the for statement, and the yield statement.
We’ll cover more data structures in More Data Collections. We’ll look at the set in Collecting Items : The set. Mappings : The dict describes mappings and dictionaries. We’ll use the map and sequence structures in Defining More Flexible Functions with Mappings. External Data and Files covers the basics of files. Files II : Some Examples and Some Modules covers several closely related operating system ( OS ) services. Files III : The Grand Unification presents some additional material on files and how you can use them from Python programs.
Organization and Structure. Data + Processing = Objects describes the object-oriented programming features of Python. Objects: A Retrospective reviews objects we’ve already worked with. Then we can examine the basics of class definitions in Defining New Objects. In Inheritance, Generalization and Specialization we’ll introduce a very significant technique for simplifying programs. Additional Classy Topics describes some more tools that help simplify class definition.
We’ll take a first look at how we can write classes that look like Python’s built-in classes in Special Behavior Requires Special Methods. New Kinds of Numbers: Fractions and Currency shows how we can build very useful kinds of numbers. We can create more sophisticated collections using the techniques in Creating New Types of Collections.
Modules : The unit of software packaging and assembly describes modules, which provide a higher-level grouping of class and function definitions. It also summarizes selected extension modules provided with the Python environment. Module Definitions – Adding New Concepts provides basic semantics and syntax for creating modules. It also covers the organization of the available Python modules. Essential Modules : The Python Library surveys the modules you’re most likely to use. We’ll look at how to handle currency in Fixed-Point Numbers : Doing High Finance with decimal. Time and Date Processing : The time and datetime Modules defines the time and calendar modules. Text Processing and Pattern Matching : The re Module shows how to do string pattern matching and processing.
Some of the commonly-used modules are covered during earlier chapters. In particular the math and random modules are covered in The math Module – Trig and Logs and the string module is covered in Sequences of Characters : str and Unicode. Files II : Some Examples and Some Modules touches on many more file-handling modules.
Fit and Finish. We finish talking about the fit and finish of a completed program in Fit and Finish: Complete Programs. The basics of a complete program are covered in Wrapping and Packaging Our Solution. Many species of programs are described in Architectural Patterns – A Family Tree.
Here is how we’ll show Python programs in the rest of the book. The programs will be in separate boxes, in a different font, often with numbered “callouts” to help explain the program. This example is way too advanced to read in detail (it’s part of Mappings : The dict) it just shows what examples look like.
combo = { }
for i in range(1,7):
for j in range(1,7):
roll= i+j
combo.setdefault( roll, 0 )
combo[roll] += 1
for n in range(2,13):
print "%d %.2f%%" % ( n, combo[n]/36.0 )
The output from the above program will be shown as follows:
2 0.03%
3 0.06%
4 0.08%
5 0.11%
6 0.14%
7 0.17%
8 0.14%
9 0.11%
10 0.08%
11 0.06%
12 0.03%
Tool completed successfully
We will use the following type styles for references to a specific Class, method() , attribute, which includes both class variables or instance variables.
Tip
Tips
There will be design tips, and warnings, in the material for each exercise. These reflect considerations and lessons learned that aren’t typically clear to starting OO designers.