In addition to the standard OS X installation that you purchase from the Mac App Store, you can use Apple's OS X Server package to convert your system into OS X Server. If you're wondering what it is for and what it will do to the system when installed, read on.
MacFixIt reader Pedro recently wrote in with questions that others interested in the server will undoubtedly have:
[With regard to Apple's Server package], can you run OS X Server on an iMac running Mountain Lion? Do you end up with two machines: the server and the computer? Do you have to switch tasks (now the server, then the iMac)? Does the installation (and operation) of the server slow the iMac to any degree? Is it perhaps better to run the server on a physically different machine (like a Mac Mini)?
How should you wire the server to the modem-router? Ethernet to the router and computers Wi-Fi between them and to the server? (I mean, is there a hierarchy like all should go first through the server?)
OS X comes with several servers installed, including those for network services like screen sharing, printer sharing, and file sharing, and also those for local application support such as the window "server" that draws windows and interface elements on the screen. There are also a number of hidden and unused server packages such as the FTP server that have been replaced with more secure or faster options, but which can be configured for use if needed.
These server packages can be used to turn an OS X client into a rudimentary server of sorts, but by default both the types of included server options and your ability to configure them are limited, primarily to that which would be most useful to home users.
When you install OS X Server, you add a number of services and enhanced configuration programs to the existing OS X installation that give you far greater ability to provide network services and that change these services' capabilities. You end up with one system that has a single installation of OS X "Server," not two separate installations and not one that can be dynamically switched between the standard client and server configurations as is needed.
The system should not be affected by the presence of the server software (just as it should not be affected by any other software packages), so you should be able to run normal OS X applications just fine without any performance hits. There are some optimizations that OS X Server will perform to better enhance the performance of the background services, but these usually do not affect how well a program runs.
Servers are generally dedicated machines for specific tasks (an e-mail server, a DHCP/Firewall server), and for stability and performance it is generally best to have a server that is not otherwise used. For example, if you have a high-traffic Web site or database server, then if you try to play fancy games on the same computer you chance timeouts and other problems with the server's Web clients. Additionally, using the server for alternative purposes may require rebooting, which will interfere with client connectivity.
In good network service infrastructures there will be automatic fallback servers that are in sync with the main server, and which will take over if the first one goes down; however, this requires a lot of configuration and purchasing of appropriate server hardware to implement properly, which may be beyond most homes and small businesses. OS X Server supports these options, allowing you to cluster servers together to enhance their reliability and performance.
With regard to where the server is placed hierarchically in the network, servers usually sit in a closet or server room of sorts and provide file sharing, printer sharing, and other centralized services to the local network. They can serve as a modem/router that establishes and controls the entire network, or can be appended to the existing network to provide side services like file sharing.
This configuration really depends on what services you are using and how you wish to have them configured. Not all services require the network to pass through them, but some do. For instance, if you wish to use the server as a firewall and DHCP server then it will have to be hierarchically "above" the rest of the network, either before the router or being itself used as the router.
As for the question of whether you need OS X Server: If you need to share a single printer then OS X Client can do this just fine (its Printer Sharing service is in essence a basic printer server), and can even be set up on a dedicated "print server" computer to share one or more printers on the network. Likewise, if you need to share files, host basic Web pages, and even share your Internet connection, then an OS X client can do this with basic on/off and account-based access capabilities with permissions settings.
However, if you need advanced capabilities such as fine-tuning the network routing or enabling various Web service plug-ins (such as WebDAV, authentication, and certificate management), or if you need additional services like calendaring, VPN, centralized authentication via Open Directory, and databases like MySQL, then the OS X Server will provide this capability. You can always tweak an OS X client system and install server packages on it in order to turn it into a server, but Apple's Server package does this in one easy installation and provides the tools necessary to easily administer the server, instead of having to get under the hood with the command line and configuration files.
My recommendation for anyone considering the Server package is to first see if the regular OS X client can do what you need. If not, the main services that OS X Server offers are enhanced file-sharing that makes it easy to set up shares and access permissions for large work groups; centralized accounts that can be managed from the server (for example, you can block someone's access to all your computers by disabling it on the server); and acting as a backup destination for all your Macs using Time Machine, similar to a Time Capsule.
In addition to these services, OS X Server can act as an e-mail server where you create your own e-mail accounts, and also be a location to store and sync calendar and contact information for use in Apple's calendar and other calendar and address book clients. With these services you can centralize calendars and services for your organization that everyone will have access to. You can also set up a VPN connection to access your data securely from remote locations, and provide various ways to authenticate users for access to these services.
In essence, if you would like to centralize the management of any of these services for multiple computers and provide more dynamic means of accessing these services, then it's likely you need a server. You don't necessarily need to use Apple's Server package: since OS X is Unix-based, it can be configured with other server software, much of which is free and open-source, but this usually requires advanced server and Unix knowledge.