Exploring Expect

Expect is quickly becoming a part of every UNIX user's toolbox. It allows you to automate telnet, ftp, passwd, rlogin, and hundreds of other applications that normally require human interaction. Using Expect to automate these applications will allow you to speed up tasks and, in many cases, solve new problems that you never would have even considered before.

For example, you can use Expect to test interactive programs with no changes to their interfaces. Or wrap interactive programs with Motif-like front-ends to control applications by buttons, scrollbars, and other graphic elements with no recompilation of the original programs. You don't even need the source code! Expect works with remote applications, too. Use it to tie together Internet applications including telnet, Archie, ftp, Gopher, and Mosaic.

Don Libes is the creator of Expect as well as the author of this book. In Exploring Expect, he provides a comprehensive tutorial on all of Expect's features, allowing you to put it immediately to work on your problems. In a down-to-earth and humorous style, he provides numerous examples of challenging real-world applications and how they can be automated using Expect to save you time and money.

Expect is the first of a new breed of programs based on Tcl, the Tool Command Language that is rocking the computer science community. This book provides an introduction to Tcl and describes how Expect applies Tcl's power to the new field of interaction automation. Whether your interest is in Expect or interaction automation or you simply want to learn about Tcl and see how it has been used in real software, you will find Exploring Expect a treasure trove of easy-to-understand and valuable information.

• Preface
• Expect — Why another tool?
• Tcl — A Little History
• Acknowledgments
• We'd Like to Hear From You
• How To Read This Book
• Notational Conventions
• Exercises

1. Introduction — What Is Expect?
• Ouch, Those Programs Are Painful!
• A Very Brief Overview
• A First Script — dialback
• Total Automation
• Differing Behavior When Running Non-Interactively
• Partial Automation
• Dangerous, Unfriendly, Or Otherwise Unlikable User Interfaces
• Graphical Applications
• A Little More About Tcl
• Job Control
• Background Processes
• Using Expect With Other Programs
• Using Expect On UNIX
• Using Expect On Other Operating Systems
• Using Expect In Real Applications
• Using Expect In Commercial Applications — Legalese
• Obtaining Expect and the Examples
• Expect And Tcl Resources
• Exercises
2. Tcl — Introduction And Overview
• Everything Is A String
• Quoting Conventions
• Expressions
• Braces — Deferring Evaluation
• Control Structures
• More On Expressions
• Lists
• More Ways To Manipulate Strings
• Arrays
• Indirect References
• Handling Errors
• Evaluating Lists As Commands
• Passing By Reference
• Working With Files
• File I/O
• Executing UNIX Commands
• Environment Variables
• Handling Unknown Commands
• Libraries
• Is There More To Tcl?
• Exercises
3. Getting Started With Expect
• The send Command
• The expect Command
• Anchoring
• What Happens When Input Does Not Match
• Pattern-Action Pairs
• Example — Timed Reads In The Shell
• The spawn Command
• The interact Command
• Example — Anonymous ftp
• Exercises
4. Glob Patterns And Other Basics
• The * Wildcard
• More Glob Patterns
• Backslashes
• Handling Timeout
• Handling End Of File (eof)
• Hints On The spawn Command
• Back To Eof
• The close Command
• Programs That Ignore Eof
• The wait Command
• Exercises
5. Regular Expressions
• Regular Expressions — A Quick Start
• Identifying Regular Expressions And Glob Patterns
• Using Parentheses To Override Precedence
• Using Parentheses For Feedback
• More On The timed–read Script
• Pattern Matching Strategy
• Nested Parentheses
• Always Count Parentheses Even Inside Of Alternatives
• Example — The Return Value From A Remote Shell
• Matching Customized Prompts
• Example — A Smart Remote Login Script
• What Else Gets Stored In expect_out
• More On Anchoring
• Exercises
6. Patterns, Actions, And Limits
• Matching Anything But
• Really Complex Patterns
• Really Simple Patterns
• Matching One Line And Only One Line
• Tcl’s string match Command
• Tcl’s regexp Command
• Tcl’s regsub Command
• Ignoring Case
• All Those Other String Functions Are Handy, Too
• Actions That Affect Control Flow
• Example — rogue
• Character Graphics
• More Actions That Affect Control Flow
• Matching Multiple Times
• Recognizing Prompts (Yet Again)
• Speed Is On Your Side
• Controlling The Limits Of Pattern Matching Input
• The full_buffer Keyword
• Double Buffering
• Perpetual Buffering
• The Politics Of Patterns
• Expecting A Null Character
• Parity
• Length Limits
• Comments In expect Commands
• Restrictions On expect Arguments
• eval — Good, Bad, And Ugly
• Exercises
7. Debugging Patterns And Controlling Output
• Pattern Debugging
• Enabling Internal Diagnostics
• Logging Internal Diagnostics
• Disabling Normal Program Output
• The log_user Command
• Example — su2
• Recording All Expect Output
• Sending Messages To The Log
• About File Names
• Log And Diagnostic State
• Exercises
8. Handling A Process And A User
• The send_user Command
• The send_error Command
• The expect_user Command
• Dealing With Programs That Reprompt
• Dealing With Programs That Miss Input
• Sleeping
• Line Versus Character-Oriented And Other Terminal Modes
• Echoing
• Prompting For A Password On Behalf Of A Program
• Security And Insecurity
• Resetting The Terminal Upon Exit
• More On The stty Command
• The system Command
• Redirecting The Standard Input Or Output
• The expect_tty Command
• The send_tty Command
• Exercises
9. The Expect Program
• Expect — Just Another Program
• Invoking Scripts Without Saying “expect”
• Rewriting The #! Line
• The .exp Extension
• The -- And Other Flags
• The -c Flag
• The -f Flag
• Writing The #! Line
• The -i Flag
• The -n And -N Flags
• The -d Flag
• The -D Flag
• The -b Flag
• The - Flag
• The interpreter Command
• Exercises
10. Handling Multiple Processes
• The spawn_id Variable
• Example — chess Versus chess
• Example — Automating The write Command
• How exp_continue Affects spawn_id
• The Value Of spawn_id Affects Many Commands
• Symbolic Spawn Ids
• Job Control
• Procedures Introduce New Scopes
• How Expect Writes Variables In Different Scopes
• Predefined Spawn Ids
• Exercises
11. Handling Multiple Processes Simultaneously
• Implicit Versus Explicit Spawn Ids
• Waiting From Multiple Processes Simultaneously
• Example — Answerback
• Which Pattern Goes With Which Spawn Id
• Which Spawn Id Matched
• Spawn Id Lists
• Example — Connecting Together Two Users To An Application
• Example — Timing All Commands
• Matching Any Spawn Id Already Listed
• The expect_before And expect_after Commands
• Indirect Spawn Ids
• Exercises
12. Send
• Implicit Versus Explicit Spawn Ids
• Sending To Multiple Processes
• Sending Without Echoing
• Sending To Programs In Cooked Mode
• Sending Slowly
• Sending Humanly
• Sending Nulls
• Sending Breaks
• Sending Strings That Look Like Flags
• Sending Character Graphics
• Comparing send To puts
• Exercises
13. Spawn
• The Search Path
• Philosophy--Processes Are Smart
• Treating Files As Spawned Processes
• Opening Ttys
• Bugs And Workarounds
• Process Pipelines And Ptys
• Automating xterm
• Checking For Errors From spawn
• spawn -noecho
• Example — unbuffer
• Obtaining Console Output
• Setting Pty Modes From spawn
• Hung Ptys
• Restrictions On Spawning Multiple Processes
• Getting The Process Id From A Spawn Id
• Using File I/O Commands On Spawned Processes
• Exercises
14. Signals
• Signals
• Signals In Spawned Processes
• Notes On Specific Signals
• When And Where Signals Are Evaluated
• Overriding The Original Return Value
• Using A Different Interpreter To Process Signals
• Exit Handling
• Exercises
15. Interact
• The interact Command
• Simple Patterns
• Exact Matching
• Matching Patterns From The Spawned Process
• Regular Expressions
• What Happens To Things That Do Not Match
• More Detail On Matching
• Echoing
• Avoiding Echoing
• Giving Feedback Without -echo
• Telling The User About New Features
• Sending Characters While Pattern Matching
• The continue And break Actions
• The return Action
• The Default Action
• Detecting End-Of-File
• Matching A Null Character
• Timing Out
• More On Terminal Modes (Or The -reset Flag)
• Example — Preventing Bad Commands
• Exercises
16. Interacting With Multiple Processes
• Connecting To A Process Other Than The Currently Spawned Process
• Connecting To A Process Instead Of The User
• Example — rz And sz Over rlogin
• Redirecting Input And Output
• Default Input And Output
• Controlling Multiple Processes — kibitz
• Combining Spawn Ids In A Single -input Or -output
• Which Spawn Id Matched
• Indirect Spawn Ids
• An Extended Example — xkibitz
• Exercises
17. Background Processing
• Putting Expect In The Background
• Running Expect Without A Controlling Terminal
• Disconnecting The Controlling Terminal
• The fork Command
• The disconnect Command
• Reconnecting
• Using kibitz From Other Expect Scripts
• Mailing From Expect
• A Manager For Disconnected Processes — dislocate
• Expect As A Daemon
• Example — Automating Gopher and Mosaic telnet Connections
• Exercises
18. Debugging Scripts
• Tracing
• Logging
• Command Tracing
• Variable Tracing
• Example — Logging By Tracing
• UNIX System Call Tracing
• Tk And tkinspect
• Traditional Debugging
• Debugger Command Overview And Philosophy
• Stepping Over Procedure Calls
• Stepping Into Procedure Calls
• Where Am I
• The Current Scope
• Moving Up And Down The Stack
• Returning From A Procedure
• Continuing Execution
• Defining Breakpoints
• Help
• Changing Program Behavior
• Changing Debugger Behavior
• Exercises
19. Expect + Tk = Expectk
• Tk — A Brief Technical Overview
• Expectk
• The send Command
• An Extended Example — tkpasswd
• The expect Command And The Tk Event Loop
• The expect_background Command
• Multiple Spawn Ids In expect_background
• Background Actions
• Example — A Dumb Terminal Emulator
• Example — A Smarter Terminal Emulator
• Using The Terminal Emulator For Testing And Automation
• Exercises
20. Extended Examples
• Encrypting A Directory
• File Transfer Over telnet
• You Have Unread News — tknewsbiff
• Exercises
21. Expect, C, And C++
• Overview
• Linking
• Include Files
• Ptys And Processes
• Allocating Your Own Pty
• Closing The Connection To The Spawned Process
• Expect Commands
• Regular Expression Patterns
• Exact Matching
• Matching A Null
• What Characters Matched
• When The Number Of Patterns Is Not Known In Advance
• Expecting From Streams
• Running In The Background
• Handling Multiple Inputs And More On Timeouts
• Output And Debugging Miscellany
• Pty Trapping
• Exercises
22. Expect As Just Another Tcl Extension
• Adding Expect To Another Tcl-based Program
• Differences Between Expect And The Expect Extension In Another Program
• Adding Extensions To Expect
• Adding Extensions To Expectk
• Creating Script-less Expect Programs
• Functions And Variables In The Expect Extension
• Exercises
23. Miscellaneous
• Random Numbers
• Example — Generating Random Passwords
• The Expect Library
• Expect Versions
• Timestamps
• The time Command
• Exercises

• Appendix — Commands and Variables
• Commands And Flags
• Variables

This is an excellent tool, and an excellent book. I just wish (pun intended) that the tool was as easy to use as the book suggests.