After reading Chapter 1, you now can install your own programs, and you know your way around your server. In this chapter, you will learn how the server and the browser (client), talk to each other. Understanding how the server and the client communicate will help you build and debug your CGI programs.
In particular, you will learn about the following:
Using the Uniform Resource Identifier
First let's get some terminology straight. Requests to the server are in the form of a URI. A URI is a uniform resource indicator.
You might be familiar with the term URL, or maybe you use URN (uniform resource name). Quite honestly, there are a number of valid names for this term. The NCSA gurus who wrote the HTTP specifications use both the term URI and URL. They started out using URI, and I'm going to try and follow their convention. I will use URI throughout this book. You can substitute whatever name you are familiar with in its place.
A URI is made up of basically three fields. You probably are familiar with at least the first two parts of a URI, and all parts are discussed in detail in the following sections. A URI has the format
protocol://<domain name>/<requested file>
The first field of a URI is the Protocol field. The Protocol field specifies the Internet protocol that will be used to transfer the data between the client and the server. There are many valid Internet protocol schemes: FTP, WAIS, GOPHER, Telnet, HTTP, and more. For the purposes of this book, the only protocol you will be interested in is HTTP (Hyper-Text Transport Protocol). And, by the way, that's why the messages passed between the client and the server are called HTTP headers. HTTP is used to designate files, programs, and directories on a remote or local server.
Immediately following the protocol is a :// and then the domain name. The domain name is the machine address on the Internet of your server. This name or address is between the :// and the next forward slash (/).
Following the domain name and before the trailing forward slash is an optional :port number. If no port number is given, the default port of 80 is assumed. The port number as it relates to HTTP and CGI is explained in Chapter 3, "Using Server Side Include Commands." Briefly, the Unix server handles different services by sending messages received at different port addresses to programs registered for those ports. The default port for the HTTP daemon is 80. Other programs, such as FTP and Telnet, have different default port addresses. These system default port addresses are set in a file named services under the system directory named /etc.
The path the server uses to find your program follows the first single forward slash (/). The server checks each element of this path to determine whether a file, a program, or a directory is being requested.
An element is a section of the path, target directory, program, or file name. Each element is separated by a beginning and ending forward slash. In the following example, you can see that element1=cgibook, element2=chap2, and element3=test.html:
/cgibook/chap2/test.html
If the last element is a directory and no further elements follow, then the server will do one of three things:
If the element is a directory and there are more elements following, then the next element is checked.
Because PATH_INFO and QUERY_STRING data can be added to the URI after the target file name or program, the execution of the program or returning of the file does not occur until the entire URI is parsed. Each element of the URI is parsed until the target file name, program, or directory is found. If the next element is a file, the file is returned to the client.
If the next element is a program, the program is executed and the data it generates is returned to the client. (As long as valid response headers are generated.)
Once the target URI (file, program, or directory) is identified, the server continues looking for PATH_INFO and QUERY_STRING data. PATH_INFO is added after the target URI. Any valid text data can be added after the target URI. The PATH_INFO data is terminated by a question mark (?), as illustrated next, where PATH_INFO is more-information:
/cgibook/chap2/test.html/more-information?
Before the target URI is invoked, the environment variables' PATH_INFO and QUERY_STRING data is set. So if there are any additional elements after the target URI, then any data after the file and before a trailing question mark (?) is converted to path information and made available as environment variables.
Additional data can be appended to the URI by adding a question mark to the last element instead of a forward slash. This data then is called the QUERY_STRING and also is made available as an environment variable.
QUERY_STRING data also can be any valid text data. It begins after the PATH_INFO data, as shown in the following line of code, and is limited only by the size of the input buffer—usually, 1,024 bytes:
/cgibook/chap2/test.html/more-information?Query-name=Query-value&Q2=Joe&last=Smith
QUERY_STRING data normally follows a predefined format, which is explained in Chapter 5, Decoding Data Sent to Your CGI Program." Environment variables are covered in Chapter 6, "Using Environment Variables in Your Programs."
So what happens when someone clicks on your URI? Figure 2.1 shows the sequence of events that occur when the browser requests and the server return a Web page. Your CGI program and the Web page calling it are closely linked (pun intended).
When a link to your CGI program is activated, the browser or client generates request headers. The server receives the request headers, which include the address to your CGI program on the server. The server translates the headers into environment variables and executes your CGI program. Your CGI program must generate the required response headers and HTML for the server to return to the browser.
Figure 2.1. The client/server connection.
When is my browser my client?
First the browser/client makes a connection to the receiving program/server. The browser uses the domain name address as the phone number or address to reach the server.
Note: Remember that the server is just a computer connected somewhere at the other end of a wire. As far as the Internet is concerned, it makes no difference whether the server is in the same room or halfway around the world. There is, of course, some time delay difference between talking across the room and across the world. But think of it as similar to talking on the phone. Whether you are talking locally or across the country, you don't expect there to be any time lag in the conversation.
The browser looks up the domain name address—the information after the http:// and before the next backslash (/). In http://www.practical-inet.com/, for example, www.practical-inet.com is the domain name address.
Next, the browser sends the following request headers to the identified domain:
These are all called HTTP request headers. They identify to the server the basic information the client is requesting and what type of response can be accepted by the client. The server also takes all the headers sent by the client and makes them available to your CGI program in a format called environment variables (Chapter 6 goes into more detail).
If the calling Web page is an HTML form that is sending data to your CGI program, then that data also is included in the initial transaction.
The server looks at the first incoming header, the Method request header, and tries to find the URI. It does this by starting at its top-level server root directory and searching for a file that matches the URI listing. The server looks at each path name after the domain name looking for a valid file name.
Let's use as an example an HTTP request to describe how the server finds the correct file from the incoming request header:
http://www.practical-inet.com/cgibook/chap2/test.html/more-information
First, the server checks the element name cgibook. Then, because this is a directory, the server continues to chap2, another directory.
Next, the server finds that test.html is a file name. So the server examines the file extension. Because the file extension identifies this as a valid text type, the server begins the job of sending the requested URI back to the client.
One more thing before leaving the URI in the example—after test.html is more-information. This information is called extra path information and is saved and made available to the requested URI as an environment variable.
Now the server must respond with the response headers. The first response header is a status line, which tells the client the result of the search for the requested URI. This response can range from Success to Authorization Required or even Location Moved. If the status is Success, usually the contents of the requested URI are returned to the client/browser and displayed on the client's computer screen.
The next section discusses in further detail what the request and response headers look like and when and how they are sent.
All of your request headers, the response headers, your status lines, and other data are sent over the Internet. That always seemed like a giant mystery to me, but it is certainly part of the common gateway interface (CGI). So just how does it work?
On the Internet, the connection is made using TCP/IP connecting to a public socket over a predefined port. Did I lose you? If I didn't, you can skip this section. For everyone else—that's almost everybody, folks—I'll break that sentence down into parts so that you can make some sense of what's going on.
On the Internet, the connection is made using TCP/IP... TCP/IP stands for Transport Control Protocol/Internet Protocol. That means that the method for transporting your request for a Web page is controlled by some dry technical document that begins with RFC and defines the specifics of transferring Internet messages. (RFC stands for Request for Comments. RFCs are the means the Internet community uses to publish new ideas and protocols. Comments are accepted for up to six months after an RFC is published.) In short, your request message is bundled up into a language that every machine connected to the Net understands.
to a public socket... Think of the public socket as the Yellow Pages phone number of the server on which your Web page is located. A socket is a software network address that networked Unix machines use to talk to each other.
over a predefined port. There is a file, services, in a directory, /etc, on your server that contains the ports that are assigned for all the common services on the Internet—services such as FTP, gopher, and HTTP connection. The default port for the HTTP connection is 80. So if you see an :80 (or any other number) appended to the end of the URI you clicked on to get a Web page, you now know that's the port being used to connect the client to the server.
The topic of Internet connections seems to confuse lots of people, and it's important that you begin to grasp this concept. If you can begin to understand how the client and the server communicate, writing your CGI programs and the forms that support them will be much easier.
So I would like to present you with this analogy to help you understand this concept. Think of your server as an old-fashioned switchboard with an operator waiting for incoming calls. You probably have seen an old-fashioned switchboard in some old black-and-white films or maybe on a Saturday Night Live skit.
The operator receives a call on the switchboard and then gets the name of the person that you want to talk to.
This is what is happening over the Internet. The next time you click on a Web page, watch the transaction occur. You can see this on Netscape browsers on the bottom of the screen. The first thing that happens is a connect message: Looking up Host, like a search for a Yellow Pages phone number. Next, you should see Host contacted: Waiting for reply. This is the phone ringing at the other end, waiting for the operator to answer. Finally, you should see a reading file or a transferring data message. Just before that last message, the server—or operator—at the other end was looking up the specific file (or person, to remain in the operator analogy) you requested. Once the file is found, it is transferred back to the requesting client.
That's how it works by analogy and TCP/IP. Once the connection is made, the server receives a bunch of information in the HTTP request headers telling it what type of response is being requested. This is important to you as a CGI programmer; you will be using the headers later in the book to send back information to your client and to decode what the client wants from you.
HTTP headers are the language your browser and client use to talk to each other. Think of each of the HTTP headers as a single message. In the client and server sense, first there are a bunch of questions (which are the request headers) and then the answers to those questions (which are the response headers).
To use the operator analogy again, think of the request headers—which come from the client—as you asking to speak to Mr. Thae. The response headers can be the operator, responding with "Mr. Thae is in Room 904, I'm connecting you now." From there, if you have a good operator, the operator stays on the line and gives you the status of your connection request.
When the operator responded with "Mr. Thae is in Room 904," the caller got a Status response header. The first HTTP response header sent to any HTTP request header is a status line. The status line is made up of status codes.
The status codes in the response header tell the client how well your request for a URI went. Throughout this book, the status codes are discussed in detail; they are included in Appendix C, "Status Codes."
Here's an overview of status codes so that you can recognize them throughout the remainder of the book:
In summary, 100s are informational, 200s are success, 300s are redirection, 400s are client error, and 500s are server error status codes. Refer to Appendix C for a complete definition of the status codes.
There are two basic types of headers: request and response headers. The client makes the request of the server, and the server builds the response headers. The most common request header is the Get Method request header.
The client sends to the server several request headers defining for the server what the client wants, how the client can accept data, how to handle the incoming request, and any data that needs to be sent with the request.
The first request header for every client server communication is the Method request header. This request header tells the server what other types of request headers to expect and how the server is expected to respond. Two types of Method headers exist: The Simple Method request and the Full Method request.
The Simple Method request header is used only to support browsers that accept only HTTP/0.9 protocol because HTTP/0.9 is no longer the standard, and the Full Method request header duplicates the definition of the Simple Method request header. An explanation of the Simple Method request header is not included. The syntax of the Simple Method request header is illustrated in the following example.
The Simple Method request header is made up of two parts separated by spaces: the request type, followed by the URI requested:
Request_Method URI \n)
The most common request methods are Get, Post, and Head. The HTTP specification also allows for the Put, Delete, Link, and Unlink methods, along with an undefined extension method. Because you mainly will be dealing with the Get and Post methods, I will concentrate on them in this chapter.
Each of the request headers identifies a URI to the server. The difference between Get and Post is the effect on how data is transferred. The Head request method affects how the requested URI is returned to the client.
The next section covers the Full Request method line. This is the request header that includes the type of access (Get, Post, Head, and so on) that the client is requesting. Of all the request headers, this is the one that really makes things work. This is the request header that tells the server which Web page you want returned to the browser. Without this header, no data will be transferred to the calling client.
The Full Method request header is the first request header sent with any client request. The Full Method request line is made up of three parts separated by spaces: the method type, the URI requested, and the HTTP version number.
Here's the syntax of the Full Method request header illustrated both logically and by a syntactically correct example:
Request_Method URI HTTP_Protocol_Version \n (newline) GET http://www.accn.com/index.html HTTP/1.0
Each part of the Full Method request header is explained in the following list:
The Get method is the default method for following links and passing data on the Internet. When you click on a link, your browser is sending a Get Method request header. When you click the Submit button on a form, if the method is undefined in the Action field of the form, the Get Method request header is used to call the CGI program that handles the form data. Chapter 4, "Using Forms to Gather and Send Data," covers forms and this method of sending data in detail.
When you click on a URI, it usually is of the form
http://www.somewhere.com/filename.html
A Get Method request header is generated along with any other request header the browser might want to send. The URI is located and returned by the browser, unless an If-Modified-Since request header was sent along with the other request headers.
When the If-Modified-Since header is included in the request headers, the browser checks the modification date of the requested URI and returns a new copy only if it has been modified after the date specified.
When you click on a URI and that URI is a request for another Web page, you send a Get Method request header and lots of other headers to your server.
The second field in the first line of the request header of the Full Method request header is the requested URI. The URI tells the server what file or service is requested.
Normally, the Full Method request header is for a file on the server. When this is the case, the absolute path of the file/URI is included in the Method request header. An example Get Method request header is GET / HTTP/1.0.
Tip: Notice that an HTML file is not identified for this Get method. The default home page or starting Web page is index.html. If you're lazy like me and don't want to type in a Web page URI for the home page, make your home page index.html, and your Web server automatically goes to that page.
The format of the requested URI is the absolute path name of the server root. This sentence has always confused me, so I'm going to explain it here, so I can always remember what an absolute path name of the server root is. Let's use an example Get Method request header of /~yawp/test/env.html/:
If the target of the URI is a proxy server, it should send an absolute URI. An absolute URI includes the domain name and the full path name to the requested URI. The domain name in the following example is www.w3.org:
GET http://www.w3.org/hypertext/WWW/TheProject.html HTTP/1.0
The last field in the Full Method request header is HTTP version. The only valid values at this moment are HTTP/1.0, followed by a CRLF. If the request is for an HTTP/0.9 server, a Simple Method request header should be used. If you're interested in keeping up with the latest HTTP protocol, you can find a hypertext version of the HTTP RFC at
http://www.w3.org/pub/WWW/Protocols/HTTP1.0/draft-ietf-http-spec.html
Table 2.1 summarizes the request/response headers used by the server and client to communicate with each other. They are defined completely in the HTTP specification. I have included some of the more obscure ones. I will discuss several of the more common headers in more detail.
The most important thing to remember is that the request/response headers are the means by which your client and browser tell each other what is needed and what is available.
Request/Response Header |
Meaning |
|
Accept |
A header that tells the server what type of data the browser can accept. Examples are text, audio, images, and so on. |
|
Accept-Charset |
A header that tells the server what character sets the browser prefers. The default is US-ASCII. |
|
Accept-Encoding |
A header that tells the server what type of data encoding the browser can accept. Examples are compress and gzip. |
|
Accept-Language |
A header that tells the server what natural language the browser prefers. The default is English. |
|
Allow |
A header that tells the browser what request methods are allowed by the server. Examples are Get, Head, and Post. |
|
Authorization |
A header used by the browser to authenticate itself with the server. It usually is sent in response to a 401 or 411 code. |
|
Content-Encoding |
A header used to identify the type of encoding used on the data transfer. An example is compressed. |
|
Content-Language |
A header that identifies the natural language of the data transferred. |
|
Content-Length |
A header that identifies the size of the data transfer in decimal bytes. |
|
Content-Transfer-Encoding |
A header that identifies the encoding of the message for Internet transfer. The default is binary. |
|
Content-Type |
A header that identifies the type of data being transferred. An example header is Content-Type: text/html \n. |
|
Date |
A header that identifies the GMT date/time at which the data transfer was initiated. |
|
Expires |
A header that identifies the date/time at which the data should be considered stale. This header often is used by caching clients. |
|
Forwarded |
A header used by proxy servers to indicate the intermediate steps between the browser and server. |
|
From |
A header that should contain the Internet e-mail address of the client. This header is no longer in common use. |
|
If-Modified-Since |
A header that makes the request method a conditional request. A copy of the requested URI is returned only if it was modified after the time specified. |
|
Last-Modified |
A header that identifies the date/time when the URI was last modified. |
|
Link |
A header used for describing a relationship between two URIs. |
|
Location |
A header used to define the location of a URI. Typically, this header is used to redirect the client to a new URI. |
|
MIME-Version |
A header used to indicate what version of the MIME protocol was used to construct the transferred message. |
|
Orig-URI |
A request header used by the client to specify to the server the original URI of the requested URI. |
|
Pragma |
A header used to specify special directives that should be applied to all intermediaries along the request/response chain. This header usually is used to provide directives to proxy servers or caching clients. |
|
Public |
A header used to list the set of non-standard methods supported by the server. |
|
Referer |
A request header that identifies to the server the address (URI) of the link that was used to send the Method request header to the server. |
|
Retry-After |
A response header used to identify to the client a length of time to wait before trying the requested URI again. |
|
Server |
A response header that identifies the server software used by the server. |
|
Title |
A header that identifies the title of the URI. |
|
URI-header |
A uniform resource identifier. |
|
User-Agent |
A request header that identifies the type of browser making the request. |
|
WWW-Authenticate |
A response header required when status response headers of Unauthorized (401) or Authorization refused (411) appear. This header is used to begin a challenge/response sequence with the client. |
After the initial Method request header, one of the more common and useful request headers is the Accept request header. The Accept request header tells the server what type of response the client can handle.
The Accept request header has the following format:
Accept: media-type; quality.
The basic media types are explained in Table 2.2. The media types are of MIME format. A complete list of MIME types is included in Appendix A, "Mime Types and File Extensions."
MIME Type |
Definition |
|
Application |
Tells the server what application to run based on the file extension. |
|
Audio |
Type of audio that can be handled by the browser. Commonly includes basic, x-aiff, and x-wav. |
|
Image |
Type of image that can be handled by the browser. Commonly includes gif and jpeg. |
|
Text |
Type of text that can be handled by the browser. Commonly includes html, plain, rich text, and x-setext. |
|
Video |
Type of video that can be handled by the browser. Commonly includes mpeg and quicktime. |
The first field of the Accept request header is the type of media that can be handled by this browser. That field is followed by a semicolon and then the quality factor. The quality factor is usually a request to not send 100 percent of the data associated with the URI. Adjusting the quality factor can speed up downloads; in most cases, the quality of the sound, image, or video is greater than the quality required for viewing or listening from your computer, as illustrated here:
Accept: audio/*; q=0.5
This means that I can accept any type of audio, and please degrade the audio data by 50 percent. Degrading the audio means less data transfer. This can be used to speed up audio transfers—for example, when you are receiving only voice and don't care about full-quality sound.
The * in this example can be used on either side of the media-type designator. The default for the Accept media type is */*. Because the Accept header should be used only for restricting the types of media the client can receive, Accept */* is redundant, not required, and not recommended.
The common media types are text, image, and audio. Some of the text types are HTML, plain, x-dvi, and x-c. HTML and plain are the standard text media types used on the Net. For image, JPEG and GIF are the two standards right now. JPEG is becoming the new preferred image format because of its smaller data size.
If you are not concerned about losing some detail, the Quality field can be used to speed up the downloading of files. The image format JPEG is an example in which a degradation in data, by removing detail, produces an image that is almost as good as the original and much smaller in data size. Because a large portion of the Net is connected by limited speed connections (modems and such), data transfer always should be considered when developing your Web page.
The default quality factor is 1, which translates to 100 percent. The format is q=factor. The factor can be any number from 1 to 0, and usually is expressed in tenths. An example is q=0.8.
The Get Method request header and Accept request header are the most common request headers. Your browser may send more information to the server, but these two define to the server what the request is and the fundamentals of how to respond to your request.
After the server receives the request headers, it begins to generate the correct response. The server starts by looking up the URI in the Get Method and then generates the response headers. The Get Method request tells the server what URI is desired. The other request headers tell the server how to send the data back to the client. The Accept request header with its Quality field, for example, tells the server how much to degrade the returned data.
So, in short, the response headers are the server's response to the client's URI request. This is the operator's chance to tell you to take a flying leap or to politely satisfy your every request.
In this case, you assume that you have a polite operator and a valid request. In Chapter 7, "Building an Online Catalog," you will deal with some of the more persnickety operators—the kind who want to know your user name, password, and other stuff like that.
After the server receives a request, it must choose a valid response. It starts with a response status line. This line gives the protocol version, followed by a status code. The format of a response status line follows:
PROTOCOL/Version_Number Status_Code Status_Description
The only valid protocol right now is HTTP, and version 1.0 is the standard at the moment. Notice how I add all those qualifiers; the Net moves so fast that fixed rules are sure to be overrun by some wild-and-crazy new idea. Of course, that's what makes the Net so neat.
Figure 2.2 shows the response headers generated when the server receives a Get Method request header.
Figure 2.2. The server response headers to a Get Method request header.
Let's take a moment to go through these response headers. These are the basic ones that will be returned from almost any request header.
The status response line follows:
HTTP/1.0 200 OK
Nothing to write home about in this response header. Nice, simple, and straightforward. The HTTP version number is 1.0. The status is 200. The status description is OK. This means your server found your requested URI and is going to return it to the browser.
The next line is the Date response header:
Date: Mon, 02 Oct 1995 11:11:32 GMT
This is the time at which the server generated the response to the request header. The date must be in Greenwich Mean Time (GMT). The date can be in one of three formats, described in Table 2.3.
Format Example |
Format Description |
|
Sun, 06 Nov 1995 06:15:10 GMT |
Originally defined by RFC 822 and updated by RFC 1123, this is the preferred format Internet standard. |
|
Sunday, 06-Nov-95 06:15:10 GMT |
Defined by RFC 850 and made obsolete by RFC 1036, this format is in common use, is based on an obsolete format, and lacks a four-digit year. |
|
Sun Nov 6 06:15:10 1995 |
This is the ANSI standard date format represented in C's asctime() function. |
Only one Date response header is allowed per message, and because it is important for evaluating cached responses, the server always should include a Date response header. Cached responses are beyond the scope of this book, but, in short, can be part of a request/response chain used to speed up URI transfers.
The Server response header field contains information about the server software used to create the response:
Server: Apache/0.8.9
If you are having problems with your CGI working with a particular site, this can identify the type of server software with which your CGI is failing.
The Content-Type header field tells your browser what type of media is appended after the last response header:
Content-type: text/html
Media types are defined in Appendix A, "Mime Types and File Extensions."
The Content-Length header field indicates the size of the appended media in decimal numbers, in 8-bit format (referred to in the HTTP specification as octets):
Content-length: 1529
This header often is used by the server to determine the amount of data sent by the client, when posting form data.
Because you are passing a file URI that is a text/html type, the Last-Modified field is the time the file was last modified. This field is used for caching information:
Last-Modified: Mon, 04 Sep 1995 17:42:40 GMT
In the case in which an If-Modified-Since request header was sent, it is used in determining whether the data should be transferred at all.
The last line of the response headers is blank, and after that, the requested URI is shipped to the client. This is the blank line in Figure 2.2 just before the opening HTML tag.
This is one of the most common reasons for response headers not working. Don't make this CGI newbie mistake. All your HTTP response and request header chains must end with a blank line.
The last print statement of an HTTP header program you write should print a blank line:
print "Last-modified: $last_modified_variable\n\n";
Notice in this example that two newlines (\n) are printed. One always is required for every HTTP header, but the second newline (\n) indicates to the sever or client the end of any incoming or outgoing HTTP headers. Everything after that first blank line is supposed to be in the format defined by the Content-Type header.
So now you know all about request and response headers. You know that the browser and the server use them to transfer data between themselves. So now that you know about request/response headers, what you can do with that knowledge?
Certainly there are all types of choices, but here is a real-world example that you just might have to deal with.
One of the things I do to make a living is build Web pages. One of the most frustrating experiences I have is building a great-looking Web page that uses all the great features of HTML+ and then hearing from my customer that his Web page looks awful. What happened? Well, the most common problem is that my client did not have the latest and greatest Netscape version. The browser he is using just doesn't deal with the latest HTML enhancements.
That's the pits. My view of the page is great. He thinks it stinks. I'll never convince him that what is out there looks good. And to him, it certainly doesn't. Have you ever seen table data when your browser doesn't support tables? UGLY!!
So what do I do about it? Well, I don't experience that frustration any more. I build two Web pages: one for the browser that handles the latest HTML enhancements and one for browsers that don't.
This means more work for me, but a more versatile page for my clients. It's not too difficult a task to take advantage of the incoming request headers and then send back a Location response header that redirects the client to the correct page for his browser. Just to show what a difference this can make, the next two figures show an HTML+ page with table data. Figure 2.3 shows the data when it is understood by the browser. Figure 2.4 shows the same page when the browser doesn't handle tables. Notice that the table data of county line locations shown in Figure 2.3 is a jumbled list at the bottom of the Web page on Figure 2.4. And finally, Figure 2.5 shows that page rebuilt without tables.
Figure 2.3. An HTML+ page for county line barbecue working.
Figure 2.4. An HTML+ page for county line barbecue broken.
Figure 2.5. An HTML 1.0 page for county line barbecue.
If you're curious, the difference between HTML+ tables and HTML 1.0 can be seen in Figures 2.3 and 2.5. Listing 2.1 is the HTML fragment for Figure 2.3. Listing 2.2 is the same data reformatted for HTML 1.0, as shown in Figure 2.5. My main complaint with list data formatting is that I can't get enough data on a computer screen. There is just too much wasted space in the HTML 1.0 version. There are other options, but none of them present the data as neatly formatted as the HTML+ tables.
<h1 > <a name="loc"> The County Line Locations </h1> <center> <table border=10 cellpadding=10 width=100%> <th align=center> New Mexico <th align=center> Austin, Texas <th align=center> Texas <th align=center> Louisiana <tr> <td align=left> <a href="New-Mexico-albq-e.html"> Albuquerque East</a> <td align=left> <a href="Austin-hill.html"> On the Hill </a> <td align=left> <a href="Texas-corpus.html"> Corpus Christie </a> <td align=left> <a href="Louisiana-new-orleans.html"> New Orleans </a> <tr> <td align=left> <a href="New-Mexico-albq-n.html">Albuquerque North </a> <td align=left> <a href=" Austin-lake.html "> On the Lake </a> <td align=left> <a href=" Texas-dallas.html "> Dallas </a> <td align=left> <a href="Louisiana-new-orleans-dtwn.html"> New Orleans Dwtn </a> <tr> <td align=left> <a href=" New-Mexico-sante-fe.html"> Santa Fe</a> <td align=left> <a href=" Austin-sixth.html "> On Sixth Street </a> <td align=left> <a href=" Texas-houston.html "> Houston</a> <td align=left> <a href="Louisiana-baton-rouge.html">Baton Rouge </a> <tr> </table>
Once you see how easy it is to direct the browser to the correct Web page, you'll agree that this is a reasonable solution, even if it does require extra work. In addition, it isn't that difficult to create a second Web page for the HTML 1.0 browsers. The HTML 1.0 fragment in Listing 2.2 shows the changes required to reformat the Web page to HTML 1.0 lists.
Listing 2.2. An HTML 1.0 fragment using lists to present county-line locations.
<h1 > <a name="loc"> The County Line Locations </h1> <h3> Austin, Texas </h3> <ul> <li><a href="Austin-hill.html"> On the Hill </a> <li><a href=" Austin-lake.html "> On the Lake </a> <li><a href=" Austin-sixth.html "> On Sixth Street </a> </ul> <h3>Texas </h3> <ul> <li><a href="Texas-corpus.html"> Corpus Christie </a> <li><a href=" Texas-dallas.html "> Dallas </a> <li><a href=" Texas-houston.html "> Houston </a> </ul> <h3> New Mexico </h3> <ul> <li> <a href="New-Mexico-albq-e.html">Albuquerque East </a> <li> <a href=" New-Mexico-albq-n.html">Albuquerque North </a> <li> <a href=" New-Mexico-sante-fe.html">Sante Fe </a> </ul> <h3> Louisiana </h3> <ul> <li><a href="Louisiana-new-orleans.html"> New Orleans </a> <li><a href="Louisiana-new-orleans-dtwn.html"> New Orleans Dwtn</a> <li><a href="Louisiana-baton-rouge.html">Baton Rouge </a> </ul>
The following section describes the steps required to test for the browser type and then send back the correct HTTP response headers to the server.
Your CGI program will test for the browser type and then generate a Location response header. The Location response header tells the browser/client to get the Web page from a different location. The browser will get the correct Web page, and your Web client will never see an UGLY-looking page.
How can you tell which browser is accessing your Web page? Well, the server does a lot of initial work for you.
The server is a wonderful, overworked, underpaid machine. One of the great things that it does for you is convert a lot of the useful header fields into environment variables. The server converts the User-Agent request header into the environment variable HTTP_USER_AGENT.
The Perl script in Listing 2.3 uses the HTTP_USER_AGENT environment variable to determine the browser type and then return an HTTP Location command to point the client to the correct Web page.
Note: Perl is a really fantastic, easy-to-use, easy-to-learn scripting language. It also can be very cryptic. It has lots of special predefined variables that you can use to shorten your code and make it more efficient. In general, I won't use those shortcuts in this book, and I often don't use them in my own code.
DON'T use cryptic variable names.
The CGI program to determine which browser is calling your Web page has two basic steps. First, it must figure out which browser is accessing it. Then, it must return the correct location headers based on the information figured out in step one.
Because Netscape is the offending browser by going off on its own and implementing all those cool extensions that are so much fun to use, let's just deal with the Netscape browser. If Netscape were the only browser that could handle tables, this program would be complete. In practice, this code should deal with all the browsers that can and can't handle the HTML+ extensions.
The format of HTTP_USER_AGENT is illustrated by how these two popular browsers define their User-Agent request header:
You can find out what types of browsers are looking at your Web page by looking in the server log files. These log files are discussed in further detail in Chapter 10, "Keeping Track of Your Web Page Visitors."
The easiest thing to do is to split HTTP_USER_AGENT into fields and then compare them against browsers you know will work for your enhanced Web page. Listing 2.3 contains the Perl code to do this. As with all the code in this book, I step through the new and relevant Perl code. You are not expected to know Perl. However, I hope you will feel comfortable enough with Perl by the time you complete this book to write CGI programs of your own.
01: #!/usr/local/bin/perl
02
03: @user_agent = split(/\//,$ENV{'HTTP_USER_AGENT'});
04
05: if ($user_agent[0] eq "Mozilla"){
06: @version = split(/ /,$user_agent[1]);
07: $version_number = substr($version[0], 0, 3);
08: if ($version_number < 1.1){
09: print "Location: http://www.county-line-bbq/clbbq-plus.html.com\n\n";
10: }
11: else{
12: print "Location: http:// www.county-line-bbq/clbbq-minus.html.com \n\n";
13: }
14: }
15: else{
16: print "Location: http:// www.county-line-bbq/clbbq-minus.html.com \n\n";
17: }
It takes several steps to get the data in the HTTP_USER_AGENT environment variable into a format your CGI program can use. First, you need to separate out the browser type. This is the part of the HTTP_USER_AGENT field before the first forward slash (/).
Line 3 uses the split function to separate the HTTP_USER_AGENT variable into parts wherever it finds a /. The split function in Perl is really powerful, and because each portion of line 3 is important and possibly new to you, each element of line 3 is explained in detail in the following list:
So now you have the name of the browser in the first element of the @user_agent array. The next thing to do is find out which browser is calling you.
Line 5
if ($user_agent[0] eq "Mozilla"){
compares the first element of the array @user_agent with the string Mozilla. If they match, you take the if path. If they don't, you take the else path. The CGI program uses the comparison operator eq because it is comparing strings instead of numbers. In Perl, strings are compared with eq and numbers with ==.
The next thing to do is to figure out what version of the browser is accessing your Web page. Even Netscape couldn't read HTML tables before version 1.1. So you need to look at the rest of the data in the @user_agent array and separate that out to get the version number.
Line 6
@version = split(/ /,$user_agent[1]);
examines the second field returned from the last split command and splits it based on any spaces it finds.
So now the first field in the @version array, $version[0], should contain the Mozilla version number 1.1N. The next step is to turn this into a number so that you can decide whether it is version 1.1 or greater.
The version returned from the split function includes an ASCII character in it—the N, to be exact. This means the program can't compare it against a number. If you leave the N in the version, the code must check for every version of Netscape because string comparison is an exact match, unlike numbers that you can compare against a range. A string comparison would require the code to check for version 1.1N, 1.0N, 1.0B, and so on.
If you turn the version into a number, the code can look for all versions that are earlier than version 1.1. Version 1.1 of Netscape is the first version number that handles tables.
Examine line 7:
$version_number = substr($version[0], 0, 3);
Now the CGI program can check for old Mozilla version numbers.
Line 8,
if ($version_number < 1.1){
shows that any Mozilla version that is equal to or greater than 1.1 will pass this test. Notice that this is a numeric test against something removed from a string. That's what makes Perl so popular. It does the right thing, even for me.
That completes step one: finding out what type of browser is calling your Web page. Now all the code has to do is tell the browser which Web page you really want it to access.
This part is amazingly straightforward! Just print the Location response header with the URI of the correct Web page.
Lines 9 through 16 print the correct headers. Line 9,
print "Location: http://www.county-line-bbq/clbbq-plus.html.com\n\n";
redirects the client to the HTML+ enhanced page.
Line 12,
print "Location: http:// www.county-line-bbq/clbbq-minus.html.com ";
redirects the client to the HTML 1.0 page.
Before the response headers are sent to the browser, the server steps in and generates any additional required response headers.
The program told the server that it wanted the browser to go to a different location. The server parsed the response header's output and added the required response headers for me. In particular, the first header of every response message must be a Status response header. In this case, that means a Status header giving the client a redirection response such as the following:
HTTP/1.0 302 Redirection
Then the Location command is included in the response headers, and the client then goes to the correct location.
Now your browser will retrieve the correct Web page for its capabilities. I will continue to refer to the HTTP headers throughout this book. This is just one simple example of how they can be used to make your Web pages more effective for your clients. In Chapter 7, where you put everything together, you will see HTTP headers as part of a complete on-line catalog application.
This chapter introduced you to client/server architecture. The browser and your CGI program are a classic example of the client\server architecture. The client requests some service of your CGI program. Your CGI program, the server, responds or services the client's request.
You also learned that the request and response system is initiated using HTTP headers. These headers are called request/response headers. The HTTP request/response headers are sent through the Internet using the TCP/IP message protocol.
The first header of every HTTP request/response sequence is the Method request header. And the first response header always will be a Status response header. The Method response header defines what the server is expected to do with any additional data and how that data might affect the URI in the Method response header. The Status response header from the server defines the success or failure status of the Method response header.
This basic knowledge is the foundation for many future applications—one of which is redirecting your Web page client based on the User-Agent HTTP header. Tomorrow you will learn the fundamentals of how to build an interactive Web site. In Chapter 3, you will learn all the details you need to know to implement Server Side Include commands, which enable you to build interactive Web pages with very little programming knowledge. In Chapter 4, you learn how to send data to your CGI program, the basis for making any interactive CGI application.
Q: What are the basic headers required for returning a Web page?
A: The question seems to boil down to what do you have to do to return HTML from you CGI program. The answer is not very much!
First and most common is the Content-Type response header. Use this when your CGI program is going to return some MIME-compliant data. Remember that the Content-Type header tells the browser what type of data to expect so that it can launch the proper application to receive it. The server will do any remaining work required to go with the returned data.
Next, you could send a Location response header. The browser will receive along with the Location response header a Status response header of 301, telling the browser about the moved URI. Your server generates the Status response header. The Location response header tells the browser that the request URI is at another location.
Finally, your CGI code could return one of the many status codes describing to the browser the status of the URI request. If you do this, you need to return the Status response header from a non-parsed header (NPH) CGI program. The NPH-CGI program doesn't get any help generating response headers from the server. If your program is generating the Status response header, however, you don't want help from the server because the server's response will conflict with your Status response header. NPH-CGI programs are discussed in Chapter 4.
These three response headers—Content-Type, Location, and Status—are the basic response headers that your CGI program will use to return information to your client.
One Last Note:
Always Always Always remember to sent two newlines (\n) after outputting the last response header from your CGI program. This is such an easy thing to do and is so often the source of broken CGI programs.
Q: How did you get that screen capture of the response header in Figure 2.2?
A: This one is kinda easy and therefore fun to play with. Remember that section on TCP/IP and how the connection is to a public socket over a predefined port? Well, that port for the HTTP server is number 80. So if you first log into your server, you can then telnet to port 80.
Take a look at the way I did this in Figure 2.2.
First I did a regular telnet connection to my Internet provider. Once I was logged into my provider's Unix machine, I telneted to one of the Web servers I'm responsible for. I did this from the command line by typing > telnet www.accn.com 80.
The 80 also could be replaced with HTTP. HTTP is the name of the program or daemon that is assigned to listen for and interpret connections on port 80. The default port for HTTP's Internet connection is 80. Using 80 in this command will always work. Using HTTP usually works.
Next, I just typed a valid Get Method request header. I could have requested a CGI program. I even could have sent PATH INFO and QUERY_STRING data. This is a great way to see what the server does with your request headers.
You can send as many valid request headers as you want this way; just end the sequence of request headers with a blank line. The server will process the typed request headers just as if it had received them in the "normal" TCP>IP manner. As far as the server is concerned, it has received the request headers in a normal manner. It can't tell that these request headers were typed from the command line.
Gook luck and have fun with this one. It's a great learning tool!
Q: There seem to be a lot of HTTP headers. How do you tell the request headers from the response headers?
A: Well, for the most part, you can't. Remember that HTTP headers can be used as both client and server HTTP headers. There are a few headers that describe just the server; these are always response headers. The other headers can be used as both response and request headers, however. Think of the Content-Length header. This header is used by both the client and the server for most transactions. When the client is sending Post data, a Content-Length request header is sent to the server. When the server is returning an HTML file, a Content-Length response header is sent to the client.
As you can see, whether an HTTP header is a request or response header is based on the sender. Request headers are sent by the client. Response headers are sent by the server.