The try, except, finally and raise statements
A well-written program should produce valuable results even when exceptional conditions occur. A program depends on numerous resources: memory, files, other packages, input-output devices, to name a few. Sometimes it is best to treat a problem with any of these resources as an exception, which interrupts the normal sequential flow of the program.
In Exception Semantics we introduce the semantics of exceptions. We’ll show the basic exception-handling features of Python in Basic Exception Handling and the way exceptions are raised by a program in Raising Exceptions.
We’ll look at a detailed example in An Exceptional Example. In Complete Exception Handling and The finally Clause, we cover some additional syntax that’s sometimes necessary. In Exception Functions, we’ll look at a few standard library functions that apply to exceptions.
We descibe most of the built-in exceptions in Built-in Exceptions. In addition to exercises in Exception Exercises, we also include style notes in Style Notes and a digression on problems that can be caused by poor use of exceptions in A Digression.
An exception is an event that interrupts the ordinary sequential processing of a program. When an exception is raised, Python will handle it immediately. Python does this by examining except clauses associated with try statements to locate a suite of statements that can process the exception. If there is no except clause to handle the exception, the program stops running, and a message is displayed on the standard error file.
An exception has two sides: the dynamic change to the sequence of execution and an object that contains information about the exceptional situation. The dynamic change is initiated by the raise statement, and can finish with the handlers that process the raised exception. If no handler matches the exception, the program’s execution effectively stops at the point of the raise.
In addition to the dynamic side of an exception, an object is created by the raise statement; this is used to carry any information associated with the exception.
Consequences. The use of exceptions has two important consequences.
First, we need to clarify where exceptions can be raised. Since various places in a program will raise exceptions, and these can be hidden deep within a function or class, their presence should be announced by specifying the possible exceptions in the docstring.
Second, multiple parts of a program will have handlers to cope with various exceptions. These handlers should handle just the meaningful exceptions. Some exceptions (like RuntimeError or MemoryError) generally can’t be handled within a program; when these exceptions are raised, the program is so badly broken that there is no real recovery.
Exceptions are a powerful tool for dealing with rare, atypical conditions. Generally, exceptions should be considered as different from the expected or ordinary conditions that a program handles. For example, if a program accepts input from a person, exception processing is not appropriate for validating their inputs. There’s nothing rare or uncommon about a person making mistakes while attempting to enter numbers or dates. On the other hand, an unexpected disconnection from a network service is a good candidate for an exception; this is a rare and atypical situation. Examples of good exceptions are those which are raised in response to problems with physical resources like files and networks.
Python has a large number of built-in exceptions, and a programmer can create new exceptions. Generally, it is better to create new exceptions rather than attempt to stretch or bend the meaning of existing exceptions.
Exception handling is done with the try statement. The try statement encapsulates several pieces of information. Primarily, it contains a suite of statements and a group of exception-handling clauses. Each exception-handling clause names a class of exceptions and provides a suite of statements to execute in response to that exception.
The basic form of a try statement looks like this:
except exception 〈 , target 〉 : suite
Each suite is an indented block of statements. Any statement is allowed in the suite. While this means that you can have nested try statements, that is rarely necessary, since you can have an unlimited number of except clauses on a single try statement.
If any of the statements in the try suite raise an exception, each of the except clauses are examined to locate a clause that matches the exception raised. If no statement in the try suite raises an exception, the except clauses are silently ignored.
The first form of the except clause provides a specific exception class which is used for matching any exception which might be raised. If a target variable name is provided, this variable will have the exception object assigned to it.
The second form of the except clause is the “catch-all” version. This will match all exceptions. If used, this must be provided last, since it will always match the raised exception.
We’ll look at the additional finally clause in a later sections.
The except statement can’t easily handle a list of exception classes. The Python 2 syntax for this is confusing because it requires some additional () around the list of exceptions.
except ( exception, ... ) 〈 , target 〉 :
The Python 3 syntax wil be slightly simpler. Using the keyword as will remove the need for the additional () around the list of exceptions.
except exception, ... as target
Overall Processing. The structure of the complete try statement summarizes the philosophy of exceptions. First, try the suite of statements, expecting them work. In the unlikely event that an exception is raised, find an exception clause and execute that exception clause suite to recover from or work around the exceptional situation.
Except clauses include some combination of error reporting, recovery or work-around. For example, a recovery-oriented except clause could delete useless files. A work-around exception clause could returning a complex result for square root of a negative number.
def avg( someList ): """Raises TypeError or ZeroDivisionError exceptions.""" sum= 0 for v in someList: sum = sum + v return float(sum)/len(someList) def avgReport( someList ): try: m= avg(someList) print "Average+15%=", m*1.15 except TypeError, ex: print "TypeError:", ex except ZeroDivisionError, ex: print "ZeroDivisionError:", ex
This example shows the avgReport() function; it contains a try clause that evaluates the avg() function. We expect that there will be a ZeroDivisionError exception if an empty list is provided to avg(). Also, a TypeError exception will be raised if the list has any non-numeric value. Otherwise, it prints the average of the values in the list.
In the try suite, we print the average. For certain kinds of inappropriate input, we will print the exceptions which were raised.
This design is generally how exception processing is handled. We have a relatively simple, clear function which attempts to do the job in a simple and clear way. We have a application-specific process which handles exceptions in a way that’s appropriate to the overall application.
Nested :command:`try` Statements. In more complex programs, you may have many function definitions. If more than one function has a try statement, the nested function evaluations will effectively nest the try statements inside each other.
This example shows a function solve(), which calls another function, quad(). Both of these functions have a try statement. An exception raised by quad() could wind up in an exception handler in solve().
def sum( someList ): """Raises TypeError""" sum= 0 for v in someList: sum = sum + v return sum def avg( someList ): """Raises TypeError or ZeroDivisionError exceptions.""" try: s= sum(someList) return float(s)/len(someList) except TypeError, ex: return "Non-Numeric Data" def avgReport( someList ): try: m= avg(someList) print "Average+15%=", m*1.15 except TypeError, ex: print "TypeError: ", ex except ZeroDivisionError, ex: print "ZeroDivisionError: ", ex
In this example, we have the same avgReport() function, which uses avg() to compute an average of a list. We’ve rewritten the avg() function to depend on a sum() function. Both avgReport() and avg() contain try statements. This creates a nested context for evaluation of exceptions.
Specifically, when the function sum is being evaluated, an exception will be examined by avg() first, then examined by avgReport(). For example, if sum() raises a TypeError exception, it will be handled by avg(); the avgReport() function will not see the TypeError exception.
Function Design. Note that this example has a subtle bug that illustrates an important point regarding function design. We introduced the bug when we defined avg() to return either an answer or an error status code in the form of a string. Generally, things are more complex when we try to mix return of valid results and return of error codes.
Status codes are the only way to report errors in languages that lack exceptions. C, for example, makes heavy use of status codes. The POSIX standard API definitions for operating system services are oriented toward C. A program making OS requests must examing the results to see if it is a proper values or an indication that an error occurred. Python, however, doesn’t have this limitation. Consequently many of the OS functions available in Python modules will raise exceptions rather than mix proper return values with status code values.
In our case, our design for avg() attepts to return either a valid numeric result or a string result. To be correct we would have to do two kinds of error checking in avgReport(). We would have to handle any exceptions and we would also have to examine the results of avg() to see if they are an error value or a proper answer.
Rather than return status codes, a better design is to simply use exceptions for all kinds of errors. IStatus codes have no real purposes in well-designed programs. In the next section, we’ll look at how to define and raise our own exceptions.
The raise statement does two things: it creates an exception object, and immediately leaves the expected program execution sequence to search the enclosing try statements for a matching except clause. The effect of a raise statement is to either divert execution in a matching except suite, or to stop the program because no matching except suite was found to handle the exception.
The Exception object created by raise can contain a message string that provides a meaningful error message. In addition to the string, it is relatively simple to attach additional attributes to the exception.
Here are the two forms for the raise satement.
raise exceptionClass , value
The first form of the raise statement uses an exception class name. The optional parameter is the additional value that will be contained in the exception. Generally, this is a string with a message, however any object can be provided.
Here’s an example of the raise statement.
raise ValueError, "oh dear me"
This statement raises the built-in exception ValueError with an amplifying string of "oh dear me". The amplifying string in this example, one might argue, is of no use to anybody. This is an important consideration in exception design. When using a built-in exception, be sure that the arguments provided pinpoint the error condition.
The second form of the raise statement uses an object constructor to create the Exception object.
raise ValueError( "oh dear me" )
Here’s a variation on the second form in which additional attributes are provided for the exception.
ex= MyNewError( "oh dear me" ) ex.myCode= 42 ex.myType= "O+" raise ex
In this case a handler can make use of the message, as well as the two additional attributes, myCode and myType.
Defining Your Own Exception. You will rarely have a need to raise a built-in exception. Most often, you will need to define an exception which is unique to your application.
We’ll cover this in more detail as part of the object oriented programming features of Python, in Classes . Here’s the short version of how to create your own unique exception class.
class MyError( Exception ): pass
This single statement defines a subclass of Exception named MyError. You can then raise MyError in a raise statement and check for MyError in except clauses.
Here’s an example of defining a unique exception and raising this exception with an amplifying string.
import math class QuadError( Exception ): pass def quad(a,b,c): if a == 0: ex= QuadError( "Not Quadratic" ) ex.coef= ( a, b, c ) raise ex if b*b-4*a*c < 0: ex= QuadError( "No Real Roots" ) ex.coef= ( a, b, c ) raise ex x1= (-b+math.sqrt(b*b-4*a*c))/(2*a) x2= (-b-math.sqrt(b*b-4*a*c))/(2*a) return (x1,x2)
Additional raise Statements. Exceptions can be raised anywhere, including in an except clause of a try statement. We’ll look at two examples of re-raising an exception.
We can use the simple raise statement in an except clause. This re-raises the original exception. We can use this to do standardized error handling. For example, we might write an error message to a log file, or we might have a standardized exception clean-up process.
try: attempt something risky except Exception, ex: log_the_error( ex ) raise
This shows how we might write the exception to a standard log in the function log_the_error() and then re-raise the original exception again. This allows the overall application to choose whether to stop running gracefully or handle the exception.
The other common technique is to transform Python errors into our application’s unique errors. Here’s an example that logs an error and transforms the built-in FloatingPointError into our application-specific error, MyError.
class MyError( Exception ): pass try: attempt something risky except FloatingPointError, e: do something locally, perhaps to clean up raise MyError("something risky failed: %s" % ( e, ) )
This allows us to have more consistent error messages, or to hide implementation details.
The following example uses a uniquely named exception to indicate that the user wishes to quit rather than supply input. We’ll define our own exception, and define function which rewrites a built-in exception to be our own exception.
We’ll define a function, ckyorn(), which does a “Check for Y or N”. This function has two parameters, prompt and help, that are used to prompt the user and print help if the user requests it. In this case, the return value is always a “Y” or “N”. A request for help (“?”) is handled automatically. A request to quit is treated as an exception, and leaves the normal execution flow. This function will accept “Q” or end-of-file (usually ctrl-D, but also ctrl-Z on Windows) as the quit signal.
class UserQuit( Exception ): pass def ckyorn( prompt, help="" ): ok= 0 while not ok: try: a=raw_input( prompt + " [y,n,q,?]: " ) except EOFError: raise UserQuit if a.upper() in [ 'Y', 'N', 'YES', 'NO' ]: ok= 1 if a.upper() in [ 'Q', 'QUIT' ]: raise UserQuit if a.upper() in [ '?' ]: print help return a.upper()
We can use this function as shown in the following example.
import interaction answer= interaction.ckyorn( help= "Enter Y if finished entering data", prompt= "All done?")
This function transforms an EOFError into a UserQuit exception, and also transforms a user entry of “Q” or “q” into this same exception. In a longer program, this exception permits a short-circuit of all further processing, omitting some potentially complex if statements.
Details of the ckyorn() Function Our function uses a loop that will terminate when we have successfully interpreted an answer from the user. We may get a request for help or perhaps some uninterpretable input from the user. We will continue our loop until we get something meaningful. The post condition will be that the variable ok is set to True and the answer, a is one of ("Y", "y", "N", "n").
Within the loop, we surround our raw_input() function with a try suite. This allows us to process any kind of input, including user inputs that raise exceptions. The most common example is the user entering the end-of-file character on their keyboard.
We handle the built-in EOFError by raising our UserQuit exception. When we get end-of-file from the user, we need to tidy up and exit the program promptly.
If no exception was raised, we examine the input character to see if we can interpret it. Note that if the user enters ‘Q’ or ‘QUIT’, we treat this exactly like as an end-of-file; we raise the UserQuit exception so that the program can tidy up and exit quickly.
We return a single-character result only for ordinary, valid user inputs. A user request to quit is considered extraordinary, and we raise an exception for that.
A common use case is to have some final processing that must occur irrespective of any exceptions that may arise. The situation usually arises when an external resource has been acquired and must be released. For example, a file must be closed, irrespective of any errors that occur while attempting to read it.
With some care, we can be sure that all exception clauses do the correct final processing. However, this may lead to a some redundant programming. The finally clause saves us the effort of trying to carefully repeat the same statement(s) in a number of except clauses. This final step will be performed before the try block is finished, either normally or by any exception.
The complete form of a try statement looks like this:
except exception , target : suite
Each suite is an indented block of statements. Any statement is allowed in the suite. While this means that you can have nested try statements, that is rarely necessary, since you can have an unlimited number of except clauses.
The finally clause is always executed. This includes all three possible cases: if the try block finishes with no exceptions; if an exception is raised and handled; and if an exception is raised but not handled. This last case means that every nested try statement with a finally clause will have that finally clause executed.
Use a finally clause to close files, release locks, close database connections, write final log messages, and other kinds of final operations. In the following example, we use the finally clause to write a final log message.
def avgReport( someList ): try: print "Start avgReport" m= avg(someList) print "Average+15%=", m*1.15 except TypeError, ex: print "TypeError: ", ex except ZeroDivisionError, ex: print "ZeroDivisionError: ", ex finally: print "Finish avgReport"
The sys module provides one function that provides the details of the exception that was raised. Programs with exception handling will occasionally use this function.
The sys.exc_info() function returns a 3- tuple with the exception, the exception’s parameter, and a traceback object that pinpoints the line of Python that raised the exception. This can be used something like the following not-very-good example.
import sys import math a= 2 b= 2 c= 1 try: x1= (-b+math.sqrt(b*b-4*a*c))/(2*a) x2= (-b-math.sqrt(b*b-4*a*c))/(2*a) print x1, x2 except: e,p,t= sys.exc_info() print e,p
This uses multiple assignment to capture the three elements of the sys.exc_info() tuple , the exception itself in e, the parameter in p and a Python traceback object in t.
This “catch-all” exception handler in this example is a bad policy. It may catch exceptions which are better left uncaught. We’ll look at these kinds of exceptions in Built-in Exceptions. For example, a RuntimeError is something you should not bother catching.
Exceptions have one interesting attribute. In the following example, we’ll assume we have an exception object named e. This would happen inside an except clause that looked like except SomeException, e:.
Traditionally, exceptions had a message attribute as well as an args attribute. These were used inconsistently.
When you create a new Exception instance, the argument values provided are loaded into the args attribute. If you provide a single value, this will also be available as message; this is a property name that references args.
Here’s an example where we provided multiple values as part of our Exception.
>>> a=Exception(1,2,3) >>> a.args (1, 2, 3) >>> a.message __main__:1: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6 ''
Here’s an example where we provided a single value as part of our Exception; in this case, the message attribute is made available.
>>> b=Exception("Oh dear") >>> b.message 'Oh dear' >>> b.args ('Oh dear',)
The following exceptions are part of the Python environment. There are three broad categories of exceptions.
Here are the non-error exceptions. Generally, you will never have a handler for these, nor will you ever raise them with a raise statement.
Here are the errors which can be meaningfully handled when a program runs.
The following errors indicate serious problems with the Python interepreter. Generally, you can’t do anything if these errors should be raised.
The following exceptions are more typically returned at compile time, or indicate an extremely serious error in the basic construction of the program. While these exceptional conditions are a necessary part of the Python implementation, there’s little reason for a program to handle these errors.
The following exceptions are part of the implementation of exception objects. Normally, these never occur directly. These are generic categories of exceptions. When you use one of these names in a catch clause, a number of more more specialized exceptions will match these.
Input Helpers. There are a number of common character-mode input operations that can benefit from using exceptions to simplify error handling. All of these input operations are based around a loop that examines the results of raw_input and converts this to expected Python data.
All of these functions should accept a prompt, a default value and a help text. Some of these have additional parameters to qualify the list of valid responses.
All of these functions construct a prompt of the form:
your prompt [ valid input hints ,?,q]:
If the user enters a ?, the help text is displayed. If the user enters a q, an exception is raised that indicates that the user quit. Similarly, if the KeyboardInterrupt or any end-of-file exception is received, a user quit exception is raised from the exception handler.
Most of these functions have a similar algorithm.
User Input Function
Construct Prompt. Construct the prompt with the hints for valid values, plus ? and q.
While Not Valid Input. Loop until the user enters valid input.
Try the following suite of operations.
Prompt and Read. Use raw_input() to prompt for and read a reply from the user.
Help?. If the user entered “?”, provide the help message.
Quit?. If the user entered “q” or “Q”, raise a UserQuit exception.
Other. Try the following suite of operations
Convert. Attempt any conversion. Some inputs will involve numeric, or date-time conversions.
Validate. If necessary, do any validation checks checks. For some prompts, there will be a fixed list of valid answers. There may be a numeric range or a date range. For other prompts, there is no checking required.
If the input passes the validation, break out of the loop. This is our hoped-for answer.
In the event of an exception, the user input was invalid.
Nothing?. If the user entered nothing, and there is a default value, return the default value.
In the event of any other exceptions, this function should generally raise a UserQuit exception.
Result. Return the validated user input.
Functions to implement
|ckdate:||Prompts for and validates a date. The basic version would require dates have a specific format, for example mm/dd/yy. A more advanced version would accept a string to specify the format for the input. Much of this date validation is available in the time module, which will be covered in Dates and Times: the time and datetime Modules. This function not return bad dates or other invalid input.|
|ckint:||Display a prompt; verify and return an integer value|
|ckitem:||Build a menu; prompt for and return a menu item. A menu is a numbered list of alternative values, the user selects a value by entering the number. The function should accept a sequence of valid values, generate the numbers and return the actual menu item string. An additional help prompt of "??" should be accepted, this writes the help message and redisplays the menu.|
|ckkeywd:||Prompts for and validates a keyword from a list of keywords. This is similar to the menu, but the prompt is simply the list of keywords without numbers being added.|
|ckpath:||Display a prompt; verify and return a pathname. This can use the os.path module for information on construction of valid paths. This should use fstat to check the user input to confirm that it actually exists.|
|ckrange:||Prompts for and validates an integer in a given range. The range is given as separate values for the lowest allowed and highest allowed value. If either is not given, then that limit doesn’t apply. For instance, if only a lowest value is given, the valid input is greater than or equal to the lowest value. If only a highest value is given, the input must be less than or equal to the highest value.|
|ckstr:||Display a prompt; verify and return a string answer. This is similar to the basic raw_input(), except that it provides a simple help feature and raises exceptions when the user wants to quit.|
|cktime:||Display a prompt; verify and return a time of day. This is similar to ckdate; a more advanced version would use the time module to validate inputs. The basic version can simply accept a hh:mm:ss time string and validate it as a legal time.|
|ckyorn:||Prompts for and validates yes/no. This is similar to ckkeywd, except that it tolerates a number of variations on yes (YES, y, Y) and a number of variations on no (NO, n, N). It returns the canonical forms: Y or N irrespective of the input actually given.|
Built-in exceptions are all named with a leading upper-case letter. This makes them consistent with class names, which also begin with a leading upper-case letter.
Most modules or classes will have a single built-in exception, often called Error. This exception will be imported from a module, and can then be qualified by the module name. Modules and module qualification is covered in Components, Modules and Packages. It is not typical to have a complex hierarchy of exceptional conditions defined by a module.
Readers with experience in other programming languages may equate an exception with a kind of goto statement. It changes the normal course of execution to a (possibly hard to find) exception-handling suite. This is a correct description of the construct, which leads to some difficult decision-making.
Some exception-causing conditions are actually predictable states of the program. The notable exclusions are I/O Error, Memory Error and OS Error. These three depend on resources outside the direct control of the running program and Python interpreter. Exceptions like Zero Division Error or Value Error can be checked with simple, clear if statements. Exceptions like Attribute Error or Not Implemented Error should never occur in a program that is reasonably well written and tested.
Relying on exceptions for garden-variety errors – those that are easily spotted with careful design or testing – is often a sign of shoddy programming. The usual story is that the programmer received the exception during testing and simply added the exception processing try statement to work around the problem; the programmer made no effort to determine the actual cause or remediation for the exception.
In their defense, exceptions can simplify complex nested if statements. They can provide a clear “escape” from complex logic when an exceptional condition makes all of the complexity moot. Exceptions should be used sparingly, and only when they clarify or simplify exposition of the algorithm. A programmer should not expect the reader to search all over the program source for the relevant exception-handling clause.
Future examples, which use I/O and OS calls, will benefit from simple exception handling. However, exception laden programs are a problem to interpret. Exception clauses are relatively expensive, measured by the time spent to understand their intent.