ASP 3.0 The Complete Reference
Chapter 32
Setting up a DSP Environment
It is not uncommon to find that companies involved in Web
application development have few policies and procedures in place for managing
the development process. This is primarily due to the fact that most Web
applications have started as something small and have grown slowly over time.
This usually results in the application not scaling well with increased usage
and the developers running in circles trying to fix bugs and add new features
with no clear direction.
Web applications are best developed across three distinct
environments Development, Staging and Production (DSP). Each environment must
be isolated from the other on hardware, software and data levels. This prevents
changes in one environment from disabling another and provides a platform for
excellent version control. Access to each environment must be tightly
controlled. In addition, migrating code from one environment to the next should
be restricted to administrative staff, following stringent testing.
This chapter will focus on the following issues within each
environment
Configuring hardware and software, specifically the
relationship between the application components, the servers that host them,
and the team members that interact with them.
Defining a process for promoting version through each
environment, ensuring quality of the system.
The topics discussed here are coordinated from various
aspects of the Microsoft Solutions Framework Process, Team, and Infrastructure
Models. Since this document focuses on
onfiguration issues only, we recommend reading the white papers on the
Microsoft Solutions Framework on http://www.microsoft.com/technet.
The following are configuration issues that are common to
all environments.
Two issues arise when deciding how to deploy server
hardware. First, it is important that there is a clear distinction between the
servers within each environment. This
prevents problems in one environment from affecting the others. The second
issue is separating major services within each environment. This is primarily an issue with web servers
and database servers since each requires considerable resources. On small-scale projects an exception can be
made in combining services on a single machine in the Development and Staging
environment. For example, few problems
would result from combining the Development and Staging databases into a single
server. However, in all circumstances,
the Production environment should be completely separate from the other
environments.
Controlling access to application related files and data
within each environment is an important issue.
Since shared directories are usually used to access application-related
files, there must be controls on the read and write capabilities of the
development team members. For instance, since each developer will have their
own private workspace on a development server (more about this later), it is
critical that other developers can't overwrite their work. In addition, only
authorized team members should be able to overwrite files to update versions on
the Staging and Production.
On a related note, rights to manipulate the schema of the
system databases should be restricted to key individuals who will be
responsible for managing those changes and how they are promoted through the system.
All servers must have the latest service packs and updates
applied. See Appendix B Software Check List for an overview of the latest
versions of software and tools. In
addition, it is important that all servers have configurations that are as
uniform as is practical. The intention
of this is to ensure that the developers are coding to the same configuration
of the systems that will ultimately be used to test and deploy the application.
While all servers should be optimized and tuned for the
particular services they are providing, it is especially important for the
Production servers. Server performance
can be greatly enhanced by manipulating configuration and registry settings and
limiting the services running to those that are absolutely essential.
The Development Environment is the place where the
application is actually coded. The key
factor in this environment is providing a controlled environment within which
developers can create and modify code without affecting the work of others.
Developers of web applications must avoid coding in the same
web site. Even with source code management, there is nothing to prevent
developers from overwriting each others changes or from being held up while a
bug created by someone else prevents the shared application from running
properly. For this reason, each developer must be provided with a private
"workspace" on the Web server.
This workspace is essentially a virtual website containing a full copy
of all the application code. Developers check files out of Visual Source Safe
to their workspace providing isolation from the work of other developers.
However, it is important that developers update their files frequently to be
sure they are coding against all the latest changes.
There are two options for creating web workspaces:
Providing virtual sites within the development web
server for each developer
Install Personal Web Server on the developer
workstations.
The virtual site approach is ideal for script developers
since they have the full services of IIS in their own private site. Virtual
sites can be configured in two ways:
Create a virtual site within IIS and assign it a unique
Host Header name following the PROJECTNAME_INITIALS pattern, such as
"WHITEHOUSE_FDR". This can be done within the IIS Management Console.
Since these sites are on the same IP, the names must be coordinated with local
or global DNS entries.
Create a virtual site within IIS and assign it a unique
IP address. Again, DNS can be used to map site names.
Of these, the first is the best since it does not use up IP
resources. In addition, it reduces issues if absolute paths are used in the application. DNS Services are included with Windows NT
Server
The hardware does not need to be of the same capacity as the
production server, since the load generated by the developers during coding is
minimal.
The developers shared directory should have permission
should have read only permission for other developers.
Personal Web Server (PWS) is a scaled back version of IIS
that does not require Windows NT server.
It supports Active Server Pages, script debugging, and Microsoft
Transaction Server (MTS). Because PWS was not designed to support high volume
Web sites, it does not include all the features found in Internet Information
Server, such as Microsoft Site Server Express, Index Server, and Certificate
Server.
Although PWS does not offer all the features of IIS, it is a
great platform for developing components that may be shared on IIS. For instance, if the application has a COM
object that is accessed by many ASP pages, developers working in virtual sites
may have difficulty testing changes to those ASP pages if the COM object they
reference is in a state of flux.
Microsoft Visual Source Safe provides the backbone of source
code management. All files used in the
execution of the project should be included in VSS. The following are
guidelines for using VSS within the development environment:
Developers check files out to their local machines
Developers must frequently refresh their local machine
to ensure that they are up to date with the state of the project.
Files are never checked in until the developers have
ensured that they have tested all the changes they have made including testing
of related files and components.
Developers should notify others immediately when a file
is checked in that has substantial changes.
VSS can be configured to maintain a copy of the project's
files in a Shadow directory: whenever a file is entered into or updated in
the SourceSafe database, VSS will automatically update the corresponding file
in the Shadow directory. Since the latest versions of all the files are
present, the Shadow directory can serve as a pre-staging area by mapping a
virtual site on the development web server to the directory. From here, the combined efforts of all the
developers are instantly available for review and smoke testing.
All developer workspaces should be backed up on a nightly
basis since files may remain checked out over several days. This allows developers to recover interim
changes that have not been checked into VSS.
Since components are not as portable as script code, it is
important to have documentation on all the components that are integrated with
the system and all the resources they require.
This will be the only way to be sure a full installation has been
completed.
Databases present another challenge in the Development
environment.
Generally, only a small number of developers are directly
involved in the development of database schema, therefore the need to create
isolated workspaces is not an issue.
However, the management of changes to a database is a significant issue,
primarily because once a database has been deployed into the Production
environment you cannot simply replace the old schema with a new one without
destroying the data that has accumulated.
Therefore, scripts must be maintained for promoting the schema changes
through DSP. If the developers are
working on the first version of the database, then saving incremental changes
is not important. Simply save the
schema to a file and store it in VSS when the work is complete.
Two types of scripts are generally required to perform
updates on databases. The first are
those that modify the schema or data definition. Scripts for changes to tables
and related subitems can be created automatically in SQL Server Enterprise Manager
whenever a change is made. The script is saved to a file with an incremental
file name so that all changes are tracked in order. Scripts for changes to
views and stored procedures can be generated after they are created. All
scripts should be stored in VSS immediately. Once the application is ready to
be promoted, these script files are then applied to the target database. If there is only one person is responsible
for the database development, then this is all that is required. However, if two developers are making
changes then they must coordinate their efforts and provide a broader tracking
process.
The second type of script involves procedures for
manipulating the existing data to accommodate the changes in structure. These scripts are generally written manually
and may need to be applied during the key point of the schema change
process. In fact, it may be easier to
copy all the scripts to a single file that can be run as a single batch
command.
Keep in mind, the only opportunity to test the update
procedures is when moving from Development to Staging. Be sure to backup each
database before administering updates.
Be ready to test the update scripts on the Staging environment several
times.
A copy of the entire database schema should be generated and
saved with each build.
Testing is an essential part of a successful DSP
process. In the Development
environment, each developer is responsible for testing all elements of their
work before checking files into VSS.
This is vital since the files that are checked in will be copied to the
shadow directory and other developer sites.
Once development has been determined to be complete, each developer logs
into the "smoke test" or pre-staging site that is mapped to the
shadow directory and runs down a feature checklist to be sure all expected
features are present and operational.
When this process is complete the Development Manager signs off and the
project is ready to be promoted to the Staging environment.
The VSS Label function provides and excellent way to track
the promotion of the application through the DSP environment. Promotion to the Staging Environment is
documented with a "Build" label.
This would consist of the version number and incremental number such as
"3.0.001". Promotion to the
Production environment is documented with a "Release" label such as
"Release 3.0". The current version number should be stored in an
application variable in global.asa to be used on splash and about screens.
Once the application has been promoted to staging, create a
"share" copy of the project in VSS. This allows developers to
immediately start new development, while leaving the previous version
intact. This allows developers to fix
bugs in the deployed version without the risk of introducing code from new
development.
VSS also provides a utility called keyword expansion. This
can be used to insert values into a file to track the current version of that
file and it's last modified date.
The Staging environment should be identical to the
Production environment, yet should be configured on separate machines. The
primary purpose of the Staging environment is to provide a platform for design
and content approval, bug testing, and load testing, before moving the
application to the production server. Run performance monitor during testing to
check server statistics. See "Appendix C - Performance Monitoring
Settings" for more information on configuring Performance Monitor.
Once the established criteria has been met for the
application at the Development Level, the following steps should be taken:
Copy script files from the Development shadow directory
to the Staging web server. These files
can be copied manually or sent through the VSS Web Deploy function if it has
been configured in advance.
Copy and register components from the Development web
server to the staging web server. Components can be registered directly with
regsvr32.exe or through MTS packages.
Copy current Production Database (if this is an
upgrade) to Staging database server then apply the change scripts.
Testing in the Staging environment occurs on three
levels.
First, the application must be tested to ensure that
the functionality outlined in the specifications is operational.
Second, the application must be subjected to load
testing that is 150-200% higher than the anticipated average load.
Third, a small set of users is invited to perform
usability tests.
Failure in any of these tests results in the application
returning to the development environment.
Prior to testing, the Testing Manager should prepare a
script outlining all functionality within the application. Testers will use this as their guideline for
reviewing the application. Testing
should be performed on all browser platforms and operating systems that are to
be supported. In addition, all content
should be proofread.
Using Microsoft's Web Application Stress Tool, the Testing manager
will prepare an automated test session script that runs through all the major
components of the application. This script will then be run multiple times in
parallel on multiple machines to simulate hundreds or thousand of users. It is
essential that performance monitoring be run to track the servers capacity and
performance.
While usability testing could be done on during the design
phase, it is still important that the application be reviewed in this respect
before it is deployed to production.
This ensures that all the functionality can be understood and managed
and may also indicate how much training would be necessary.
Once the app has been tested and approved, it can be
promoted to the production environment.
This is the final destination of the application and is also
the place where users will interact with it.
Therefore, it will be essential to be able to ensure that everything is
operating as expected.
Do not run a Database Server on the same machine with the
Web server
To ensure that the integrity of the application is not
compromised, direct access to the machine must be limited to administrative
personnel. In addition, it is important that the machine be configured to be
lean, secure and stable. See Appendix E - Windows NT Services for an overview
of services that are eligible for removal.
Although Web Server Logging is an important tool for
tracking who has visited a site, it can consume a lot of resources on a busy
site. Since web applications often maintain their own security status through
user logins and session tracking, Web Server Logging can be limited to the
entry/login page. Eventually the application should handle its own internal
auditing -- tracking who did what within the app.
To reduce the risk of external intrusion to the application,
do not install FrontPage Extensions on the Production server.
The process for setting up the application is exactly the
same as promoting to application to Staging, however, considerable care must be
taken when attempting this process in the Production if an update is being
performed. Here are several measures to
prevent disaster:
Backup the entire web server before copying files and components.
Backup the database before applying change.
Disable services during the update. Be sure to notify
existing users that an update will take place and that the application will be
offline.
Perform the update on off-hours. Be sure that enough time is allotted to
complete the update before the application is required.
Of course, if additional hardware can be made available, the
update to Production web server can be made to a second server offline. When complete, the new server can be swapped
with the old. Substantial changes to
the database however, will make downtime inevitable.
Be sure to set the a release label in VSS.
Perform a functionality test and load testing before going live
Be sure to run the same test script used with the Staging
server.
See Chapter xx Tuning and Optimizing ASP Application for more information
on configuring Performance Monitor.
Statistics track information on data distribution within
your system. By updating this periodically, your server can determine a plan of
attack quicker and with better throughput.
With the limited information and time available to us, AVI
has identified several areas that are in need of remediation if AME is to be a
robust application. These findings are not meant to be an exhaustive list.