This content was published by Andrew Tomazos and written by several hundred members of the former Internet Knowledge Base project.

A Web Site as a Computer Program

Welcome to the fourth installment of the IKB newsletter.

We talked last time about how a computer program is a sequence of binary data that is fed into the brain of the computer, which then takes control of it.

We also talked about the process of creating programs - and how the source for them are expressed in intermediate programming languages that can be read by both humans and computers - and how these "source code" files get converted into binary data and fed into the processor.

Something really interesting happened in the 90s when the Internet was popularized by the rise of XHTML (then HTML) programming. To get some context we will take a quick run-through the protocol that the Internet runs on.

++ INTERNET PROTOCOL ++

TCP/IP stands for the Transmission Control Protocol/Internet Protocol.

Every computing device connected to the Internet uses this protocol, by definition.

It is a free open protocol. You don't have to pay anybody a royalty to use it. A document explaining how it works is publicly available.

How it works is actually very simple. Every computer on the Internet has a number assigned to it (an IP number). When your computer wants to send data to another computer, it chops it up into small data packets, and writes the number of the destination computer on the top of them. It then broadcasts these out onto your local network. One device on your network listens for them, called a router.

When the router receives it, it reads the destination number (and hopefully ignores the content) and looks up the destination in a table - its routing table - and relays a copy of the data in the general direction of the destination.

Another router and network closer to the destination hears it and catches it. The next router looks the destination up in its table, and forwards it on.

This chain reaction continues until the packet gets to the destination.

A packet can get from one side of the planet to the other in about one fifth of a second if it goes by land, about twice that long if by satellite (no matter where the other end is it's a 46,000 mile trip), and it will get retransmitted about twenty times by twenty different third-party routers.

The sharing, of the spare time and resources, of all these strangers' computing devices to relay packets for each other - is what makes the Internet so cheap. It is also what makes it decentralized. There is a central authority that dishes out IP numbers, but they are advisory in the sense that each router doesn't check in with the central authority each time it relays a packet. If the central authority went down, the Internet would still work for the time-being.

That is one of the reasons the phone companies are scared about the Internet becoming mainstream for making voice calls. Competing with a cooperative decentralized network is like trying to start a fist fight with a swarm of bees.

++ DATA PACKETS ++

There are two ways to send packets.

The basic way is as individual packets (UDP), where each packet is not guaranteed to arrive, or to arrive in the order they were sent - but this method is very undemanding on computing resources.

The more complicated way is to setup a connection stream (TCP). By building on top of the packet-relaying system we can make a special packet that signals the start and end of a connection. We then number each packet of data in order. One computer listens for a starting packet. The other computer initiates the connection by sending it.

When the listener receives it, it starts listening for the packets in order and then reassembles the data, until it gets the ending packet. This happens in both directions. A connection can last for less than a second - or it can go on for hours.

It's like a computer-to-computer binary phone call for computer programs to talk to each other.

Every device on the Internet can do these tricks. Send packets, listen for connections and initiate new ones. It is the way we transmit binary data from one device to another over the Internet.

++ WEB BROWSER & SERVER ++

A web browser - such as Firefox, Safari, or the venerable Internet Explorer - is a computer program.

It uses TCP connections to talk to other computer programs that are running on other computers (or even the same computer). The program it talks to is not a web browser - it is called a web server.

Web browsers and web servers are completely different programs. They just happen to talk to each other. They might be running on completely different kinds of computers and operating systems -- and in fact, usually are.

Most web browsers are running on Windows.

Most web servers are running on a derivative of Unix.

The reasons why are debatable, because there are both Unix desktops and Windows servers available -- but for some reason most people choose not to use either.

Regardless of the reasons, there is no doubt that that this market trend is fact, and it has been the case for over ten years.

How does a computer that can speak or listen to the Internet from more than one program know what software should get or send a message? That's what ports are for. Your browser sends a request for information to an IP address on port 80. The server at that address listens to port 80 and ignores other requests that arrive on the net. It sends its response back to your machine on whichever port is assigned to your browser by the operating system; no other application can use this port number at the same time.

++ HTTP ++

The protocol a web browser and web server use to speak to each other is called HTTP, for HyperText Transmission Protocol. That's what the thing in front of all those web addresses means. The hypertext part is a bit of a lie. You can use HTTP to transfer any kind of data just fine, it has nothing to do with Hypertext.

HTTP uses a TCP connection stream in a certain way -- for this reason we say HTTP is built "on top of" TCP.

Now, web addresses. They are actually called Universal Resource Locators (URLs), because they point to a worldwide Internet resource on a specific computer and the protocol to use on it.

The URL "TOM://DICK/HARRY" means use the protocol TOM to open an Internet connection to the computer DICK and access resource HARRY.

So "http://ikb.cc/208" means use the HTTP protocol to talk to the computer at ikb.cc and get resource 208.

If you type that into the Address field of a web browser you will get this document. Your web browser looks up ikb.cc in the Internet phone book, gets its destination number and opens a TCP connection from your computer to it.

In accordance with the HTTP protocol - your web browser then writes the text "GET 208" down the TCP connection to the web server.

The web server program figures out that "GET 208" means you want IKB item number 208, looks it up in the database and then writes back source code over the connection. The web browser then runs it.

The data sent is usually not a plain text file, although it can be, and that will work fine and the web browser simply prints it to screen.

In fact, the data it writes back can be any type of binary data - as can the name of the resource it is requesting. The web browser may make a guess about the best way to handle it, but there are ways to give the browser specific instructions as well.

++ XHTML/CSS ++

This particular document is written in a programming language called XHTML/CSS.

The web browser understands this XHTML/CSS language and interprets it.

You see, a web browser is in fact a sort of compiler or interpreter.

It takes source code and lets it control the computer.

Usually the program it runs simply displays text on your computer and dies, so usually we think of it as "displaying" web pages. The 208 in this example is the name of the "page", and the result it sends back is an encoded "page".

This "page-based" way of thinking is fairly limited because you miss out on a lot of the more interactive possibilities once you think of the web like a set of pages from a magazine.

If you want to see the source code of a page then you just click View Source in the menus of your web browser. It then displays the raw XHTML source of what you are looking at.

You will see lots of stuff between matching brackets like "My Page". This says that the Title of the document is My Page, and the web browser displays it in the top of the window.

Every programming language has different strengths and limitations.
XHTML/CSS's strength is in visually displaying things. The hard bit is that it has to take the same source code and run it on different computing devices. These different devices have different sized screens and different fonts.

This means you can't simply say display this at this point on the screen and make it look exactly like this. Everyone is using different browser software and different computing devices with different screens and resources available. Also, some people are blind and their web browser reads it aloud to them.

The language can't just do whatever you want (at least in theory).
Imagine if we write an XHTML program that "deleted all the files on the hard drive". Its okay to run such a program on your computer, but if all you have to do is click on a bad link to make that happen its not very safe. For this reason XHTML is purposely limited in the things it can do.

++ WEB SITE AS SOFTWARE ++

As part of sending XHTML from the web server to the web browser program, the server can put web addresses (URLs) as "links" back to your web server, or to other web servers running on different computers, or even to different Internet protocols than HTTP, such that when you click on them in the web browser it will connect again and download another XHTML program to run.

Let's say that the link is back to your web server, and the protocol remains HTTP. So a link like this on this page would read "http://ikb.cc/SOMETHING". When you click this link, the web browser connects back to the same web server and sends "GET SOMETHING". Based on that, the web server gives your web browser another XHTML program to run.

Now, here is the tricky part. The web server has given the web browser a program to run that controls your computer to some extent, and as part of it, it has put a string of its chosing in it, such that when you click on it it gets that string sent back to the web server.

The string can be anything, and all you see is the link you click. The string could be "g532875tg2378u3grt287gb", it doesn't matter that you can't read it - because the web server created it and it's the web server that gets it back. It only needs to mean something to the web server. You don't need to see it.

So, if we put aside the page-based way of thinking, when you are using a web server and staying on that server - the web server is controlling your user experience in an interactive way. Your computer and web browser are acting as dumb slaves for you and the web server program.

The fact that the web server program is running on a different computer becomes, in many ways, irrelevant. All you are doing, really, is running a piece of software.

The interesting bit is that it is on a different computer.

++ PROS ++

The industry loves this idea as a software distribution system.

The pluses for vendors of distributing software as a web application in this manner are that it is centralized, and the vendor remains in control of it. The vendor can force upgrades and charge on-going fees for web software, because it is technically running on the vendor's computer and not the end-user's computer.

Maintaining different versions becomes trivial. Vendors have to install the software on only one big server computer, rather than maintaining it across thousands or millions of end users' computers.

Everybody gets upgraded at the same time whether they like it or not.

The end-user investment in starting to use a piece of software for the first time goes from the old way -- driving to a store, or downloading it over the Internet, installing it and rebooting -- to the new way -- simply typing "whatever.com" into the address bar of your web browser. Much quicker and easier.

Through the cross-platformness of web technologies, distributing your software across Windows, Mac, the Unixes, and even mobiles becomes easier. All the different web browsers and user agents have done alot of the work for you. There are different interpretations of the standards to work around, but its a lot easier than compiling a different version of your software for five different platforms and maintaining them.

That's one of the reasons Microsoft freaked out a little bit when the idea of web software came about - suddenly the choice of desktop operating system was going to become less and less relevant.

That's why Microsoft quickly decided that the web was the future and made Internet Explorer an inseperable part of Windows - because they wanted to have control of the gateway to the web, the web browser, the web software runtime environment.

Having control of that gateway, and getting in the middle between end users and web software vendors, means that they do things to protect their revenue stream if they have to and also crushes competition.

++ CONS ++

There are however downsides of web software over locally running software. If the server you are using is a long way away, there will always be a delay between a user action and the web server software finding out about it. If the server is on the other side of the planet, that delay cannot be less than a tenth of a second from simple physics, and any routers or gateways will add to the burden.

This is because the electrons can only travel at slightly less than the speed of light, and if the server is running on the other side of the world, it takes them somewhat over a hundred milliseconds to travel there and back. Studies have shown that a person will notice any lag over 6 milliseconds, so this is not a trivial point.

There is one common workaround - key software that runs the user experience is sent to the local web browser, and runs there. This might be as simple as a web form with a few text fields, as intricate as a javascript based calendar, or as complicated as an interactive flash movie or Java applet. Once the user has finished whatever complicated interactions they need to, the web browser can send the results back to the server. A sixth of a second to save a document is far less egregious than a sixth of a second delay when changing a control setting.

This fails when the software needs to perform a round trip to the server in response to a common user action. For example, both video conferencing and multiplayer gaming require a constant stream of user actions to flow between separate places. There will always be a little bit of lag, and if the users are separated by great distances, that lag will be noticeable. As said above, any lag over six milliseconds will be noticed.

Web applications often feel slow today because of how web pages are loaded. Most servers and most browsers look at each page as a brand new page to display, so as soon as they are told that a page needs to change, they reload the entire thing from scratch. This is the right thing to do when one is moving between Google search results, but if the different page is just the result of a spreadsheet cell updating, then throwing out the entire page on any change is way too much work.

In addition, there is no easy way for a server to change a page without reloading everything. In conventional applications, multiple things are often going on. In most mail programs, for example, it can check for new messages as you compose a reply to an old one. This is far more difficult to do with Hotmail or Gmail's web interfaces, as they do not have an independent program running. Further, current reload technology, as mentioned above, redraws everything during a reload.

There are solutions to this problem. Some of them require effort by the application vendor - a Java applet, for example, can do two things at once and present a UI that only redraws changed items. Others require effort by browser makers, such as optimizing the redraw of pages which have not changed by much. Still others require new protocols, such as the various push protocols that have been proposed which let a server send back information about what has changed. All of these will likely become more mainstream over the next few years.

++ SUMMARY ++

There are a number of different groups scrambling to create and distribute technology to make web software deliver as much a first-class user experience as desktop software.

We know of a variety of solutions to these problems, but it is not clear which ones are going to become popular. This depends greatly on which company's vision of the future gains market and mindshare. Sun, Adobe, Apple, Google, and Microsoft all have technologies to solve some of these problems. As users, if we vote with our feet and our dollars, we can encourage the best of them to flourish. The underlying problem - getting software distributed via a web browser and loaded in a few seconds - is well understood, but non trivial.

Back to Index