More Thoughts About Filters
(For some reason, I can't seem to get this article to show up on my home page -- it shows up on the Python Projects page fine... working on it...)
I have a confession to make, right up front. For some reason, I had a lot of trouble getting my mind around the filter concept. I think it speaks of my procedural-language background. But I finally made the conceptual breakthrough, and understood what others were trying to tell me.
My problem was partly with what wasn't being said. So, to cement my own understanding, and ensure others do not have the same problem, let me spew forth what I now believe I understand about writing filters for file-like input in Python. Please, if I get something wrong, poke me.
I'll start with an analogy. My appologies in advance.
There are three workers in a quarry, each operating a simple machine. Larry's machine takes in big rocks, and turns them into gravel. Next, Curly's machine takes in gravel, and turns it into pea-gravel. Lastly, Moe's machine takes in pea-gravel, and turns it into sand. Larry's machine is fed from a big hopper of rocks. Moe's machine outputs its sand to a big sand pit.
Moe's boss, we'll call him Shemp, says to Moe "The sand pit's getting low! We need more sand!" Moe says to Curly "Hey, moron, I need more pea-gravel!" Curly says to Larry "Hey Larry! Send me some of that gravel!" Larry pulls the handle on the hopper, dumping big rocks into his machine (one lands on his foot, he starts hopping). Larry's machine grinds for a while, and dumps out gravel. Larry says to Curly, "Ouch! Here! Ouch!" Curly shovels the gravel into his machine (poking Moe in the gut with the handle), and pushes the button. It grinds for a while, and dumps out pea-gravel. Curly says, "Hey, Moe! Lookie what I made!" Moe whacks Curly on the head with his shovel, then loads his machine with the pea-gravel and pushes the button. It grinds for a while, and dumps sand into the pit. Moe says to Shemp, "There you go, boss!" The lunch whistle blows. Larry, still hopping, backs over the edge of the sand pit and falls into the soft sand...
What's gone on here (other than classic slapstick)? We have a chain consisting of a source, three filters, and a sink. The initial command that began the process came from the end of the chain. This is called 'pull filtering'. Each stage in the chain pulls data from the stage before it. When you have a "pull" chain of filters working on file-like objects, each filter needs only a publicly-accessible file-like read interface. At a minimum, a 'read' method. Each filter object gets initialized with the filter object that preceeds it. You call the 'read' method of the last filter in the chain. It needs data, so it calls the 'read' method of the next filter up the chain. And so on up the chain until the 'read' method of an actual data source is called. This gives the first filter in the chain something to work on. It then passes the filtered data on via the return value of its 'read' method. And so on down the chain. Notice, no 'write' method was needed at all.
It was the lack of a 'write' method that was blowing my mind. Then this rock fell on my foot, and I got it...
Now, there's another kind of filtering that does use a 'write' method, and it's called 'push filtering'. In this case, the initial command that starts the process comes from the source, rather than the sink, by writing the data down the chain to the next stage. For my purposes, I'm going to be concentrating on 'pull' filtering.
3:31:41 PM 
|
|
