Several years before the Colossus in the U.K. and the ENIAC in the U.S., the Z3, built by Konrad Zuse in 1941, was crunching numbers in Germany. In a short article, the Register says the Z3 was the first programmable computer. Based on a binary floating-point number and switching system, it had all the attributes of today's computers, such as a control block, a memory, and a calculator. But it didn't have the ability to store the program in the memory together with the data because the memory was too small. It had a 64-word memory of 22 bits each and was able to handle four additions per second and to do a multiplication in about five seconds. And it was pretty big: five meters long, two meters high, and 80 centimeters wide. It was destroyed during WWII, and later rebuilt in 1960/1961.
Below is a photograph of Konrad Zuse with the rebuilt Z3 in 1961.
This photograph was picked from an extensive biography of Konrad Zuse (1910-1995), written by his son, Prof. Horst Zuse. Here are two links to the preface and to the specific page about the Z3 Computer.
The above page also contains a block diagram of the architecture of the Z3.
As you can see, the structure of the Z3 contains all the elements of a modern computer. According to this page at About.com, "Zuse used old movie film to store his programs and data for the Z3, instead of using paper tape or punched cards. Paper was in short supply in Germany during the war.".
In this other page from J. A. N. Lee about Konrad Zuse, you'll see the basic specifications of the Z3. In particular, it used "2,400 relays, 600 in the calculating and program section and 1,800 in the memory." And like in today's computers, the memory words had several components. Each 22-bit word length contained 1 bit for the sign, 7 exponential bits and a 14-bit mantissa.
The Register says that the Z3 was not used for cryptography.
Zuse's machine saw use during the war, but not as a codebreaker. Instead it was used to perform statistical analysis of the stresses on aircraft wings, and in particular, a problem known as wing-flutter. This vibration of an aircraft's wing can cause a critical instability during flight. The calculations needed to overcome this design issue were incredibly complex, and it was this problem that the Z3 solved.
About.com adds some details about the programmation of the Z3.
Konrad Zuse wrote the first algorithmic programming language called 'Plankalkül' in 1946, which he used to program his computers. (He wrote the world's first chess-playing program using Plankalkül.)
However, Wikipedia gives more precisions about the Plankalkül programming language, saying "no compiler or interpreter was available until a team from the University of Berli implemented it in 2000, five years after Zuse died."
Sources: Lucy Sherriff, The Register, June 2, 2004; Wikipedia.org; and various other web pages