Model Scoping, Forking, and Scenarios
From Traffic Analysis and Microsimulation
As explained in the Getting Started section of the Economic Analysis Guidelines, it is very important to have a clearly-defined scope at the outset of any type of analysis project. Similarly, in microsimulation projects clear scoping can help avoid problems such as:
- Misunderstandings or ambiguities regarding the goals of the modeling effort.
- Mission creep (unplanned expansion of the model that delays the delivery of results), such as unplanned enlargement of the geographical boundaries, adding more roads, or implementing advanced model features that were not originally part of the project.
- Misapplication of the model, i.e. attempting to use it at a level of detail for which it was never intended. An example is trying to use an area-wide "strategic" model to make detailed "final design" decisions.
- Inappropriate sequencing of activities. An example is starting to model "build" scenarios before the base model has been properly calibrated.
This guideline provides information that can help project managers avoid or minimize such problems through careful scoping and sequencing of microsimulation projects. The guideline includes:
- Information on how to establish and document the project scope.
- A method for identifying background changes in the road network and incorporating them into the microsimulation network.
- A discusson of the proper sequence for model scenario development (forking).
Scoping the Project
Confusion about the goals, objectives, or physical boundaries of a microsimulation project can cause delays and/or conflict between clients and consultants, or amongst the members of the analysis team. To avoid such problems the following process is recommended:
- Conduct a pre-project site visit. The project manager and other key members of the project team should visit the site together (usually as a driving tour in a van). If a consultant for the project has already been identified, it may be helpful to include one or more members of the consulting team. It is often preferable to schedule this site visit during a time when congestion can be observed. The visit should include stops to observe the existing bottlenecks and other troublesome parts of the study area. While driving the site, informally discuss the existing situation and some of the solutions that are being considered. The team can also use this visit to become familiar with any physical constraints on solving the traffic problem, such as large buildings, steep grades, waterways, or environmentally sensitive areas.
- Conduct a Model Scoping Meeting. Typically this meeting will include a somewhat larger group than the site visit. Usually this meeting will be be limited to internal agency stakeholders (and perhaps a small number of representatives of the consulting team, if appropriate). The moderator of this meeting can use the Traffic Analysis Scoping Record form to facilitate a discussion of the key aspects of the project: goals and objectives, geographical boundaries, level of detail required, the number of scenarios and time periods to be analyzed, any new land development to be considered, existing data resources that can be utilized, and new data that will need to be collected. At the end of this meeting the Traffic Analysis Scoping Record should be written up and circulated to internal stakeholders.
- Determine the Number of Models Required. Prior to the early 2000s, computational power limited the physical size of the areas that could be modeled with microsimulation. In that era most studies consisted of linear corridors less than 5 miles long. In more recent years there has been a tendency to model increasingly large and complex areas. Nowadays computer hardware and software are seldom limiting factors, but calibrating large models remains very time-consuming. One way of measuring the complexity of a model is to consider the size of its Origin-Destination (OD) matrix, which represents each location where vehicles can enter or exit the model (in modeling terminology these access points are called zones). The size of the OD matrix increases with the square of the number of traffic zones modeled: a 25 zone model has 600 OD pairs, a 50 zone model has 2450 OD pairs, a 100 zone model has 9900 OD pairs and a 200 zone model has 39,800 OD pairs. If we assume that each additional OD pair will add several minutes (or more) to the calibration process, it becomes clear that bigger is not always better. Therefore, if a study area has a location where the traffic volume naturally drops off, it is usually simpler to break the model into two parts. For example, two 15-mile corridors with 50 zones each will result in 4900 OD pairs, which is more manageable than a 30-mile corridor comprising 100 zones and 9900 OD pairs. (A rural area with relatively low traffic volumes is a good break point; conversely, avoid breaking the model at river crossings if you want to study traffic shifts between the bridges).
- Determine the Temporal Analysis Periods (TAPs). TAPs are the times of the day that will be modeled, such as the AM Peak period, PM Peak period, and any periods of special interest, such as holiday weekends for a model of a resort area. This topic is discussed in more detail in the Economic Analysis Guidelines.
- Develop the Model Tree. A model development tree (illustrated in the Model Forking section of this document) identifies all of the scenarios to be analyzed and their relationships to one another and the base model. It formally illustrates the way the model will evolve as the work progresses, and establishes the sequence of work activities to avoid premature forking of the model. The model tree should also identify the points during the modeling process where Network Audits and Peer Review will be conducted.
- Establish the Deliverables List. This list will identify all of the documents, videos, computer files, and other items that the microsimulation team is expected to produce. The Typical Microsimulation Deliverables section of this document includes a list of such items.
- Formalize the Contract. The Traffic Analysis Scoping Record and Deliverables List serve as a basis for preparing formal contractual documents between the agency and the consultant. Sharing these documents with the consultant helps assure that the consultant has a clear understanding of the project requirements and can set out the tasks, hours, schedule, and basis of payment logically in the contract. If the microsimulation modeling is being done by an internal team rather than a consultant, the team leader should formalize the tasks, hours, and timeline in a memo that is shared with the internal stakeholders.
- Conduct a follow-up site visit. Once a contractual relationship (or internal memorandum of understanding) is in place, the technical members of the project team should visit the site again. This will be a more technical visit to observe and document the existing conditions and become familiar with the study area. Typical activities will include photographing the existing configurations (lane use, signage, turn bays, etc.) and videotaping the traffic flow in congested areas. The technical team can also use this visit to discuss the way the model itself will be structured, such as the way new data will be collected, the layout of origin-destiation ZONES, and the way intersections will be modeled.
Establishing the DO MINIMUM and DO SOMETHING Scenarios
An important difference between microsimulation and most other types of traffic analysis is that simulation simultaneously evaluates the interaction of all elements of a transportation network, while most other analytical methods treat each element separately. Because of this characteristic, the design-year simulation network usually has to be adjusted to incorporate changes that are likely to occur as a result of projects and real estate developments that will occur (or have occurred) after the date represented in the base model, but prior to construction of the project that is being studied. Such changes can sometimes be ignored in other types of traffic analysis, but microsimulation will usually fail to yield satisfactory results if they are not considered.
A typical example of this issue occurs when attempting to model the freeway traffic that will exist 20 or 30 years in the future. When additional traffic is added to an existing simulation network, traffic attempting to get to the freeway can get stuck at a bottleneck on the local street system (perhaps a signalized intersection that is already operating close to capacity). As a result, any future problems with the freeway are masked by the local street problem. Therefore, to analyze the future freeway operations it becomes necessary to identify a minimal set of local street improvements that allow the traffic to reach the freeway.
As part of the model scoping process, the alternatives to be evaluated should be defined and documented on the Traffic Analysis Scoping Record form. There will always be a base model representing the existing conditions. In addition there are usually at least two models of the future traffic conditions, namely a DO MINIMUM and a DO SOMETHING option. (In most cases there will be several DO SOMETHING or BUILD options under consideration.) While these terms may appear self explanatory, in fact they raise some fundamental issues.
The DO MINIMUM scenario is the base road and traffic network and serves as the benchmark for evaluating all other potential improvements. In some cases the definition of the DO MINIMUM is straightforward: it is simply the existing network without modification (in other words a NO BUILD or DO NOTHING scenario). Nevertheless, there are at least four situations where the DO NOTHING and the DO MINIMUM are different:
- The traffic conditions can be improved inexpensively. An example is changing the traffic signal timing or implementing other low-cost traffic management measures to reduce delays.
- It is usually appropriate to use signal timing software such as Synchro or Transyt 7F to optimize traffic operations in the DO MINIMUM.
- In some cases, actions taken by other agencies or organizations may need to be considered. An example would be a large manufacturing facility that voluntarily changes its shift times to reduce congestion on nearby streets.
- The case where the area covered by the microsimulation network includes trunk highway projects other than the one that is being evaluated. The DO MINIMUM and DO SOMETHING networks should be coded to include planned improvements elsewhere in the network in the year they will be open. This information can typically be obtained from the State Trunk Highway Six-Year Program, the Transportation Improvement Program (TIP) for the relevant regional planning commission, and local government capital improvement budgets.
Published lists sometimes contain projects that are difficult to implement, so it may be necessary to contact the people who manage each agency’s capital improvement program to confirm which projects are likely to go forward. In some cases the implementation of one or more projects may be uncertain, so it may be necessary to carry out sensitivity tests to gauge their impacts on the project that is being evaluated. Additional model runs may be necessary to evaluate inter-related road projects, especially if they are competing for the same funding or are designed to compliment each other (for example, a bypass that will be built in phases).
- The case where work will be carried out regardless of whether or not the DO SOMETHING scenario is built. From time to time there are transportation network improvements that will be built no matter what happens with the project that is being evaluated. An example is an intersection improvement that a private developer is obliged to build as a condition of a Traffic Impact Analysis (TIA) approval. In this case, the improved (not the existing) intersection should be coded in the microsimulation network in both the DO MINIMUM and DO SOMETHING scenarios in the year it will be improved.
- The case where the existing network may be improved to establish a DO MINIMUM scenario that can be tested as an alternative to carrying out major DO SOMETHING improvements. An example is when an existing intersection or link is forecast to become heavily overcapacity in the near future and relatively minor improvements can be implemented to increase capacity (for example, by adding a turn lane at a traffic signal). This class of DO MINIMUM improvement is particularly important where the existing network is congested and where a literal DO NOTHING scenario would represent an unrealistically poor baseline for comparison.
- The microsimulation output can help identify intersections and links which exceed their capacity in the DO MINIMUM. One useful rule-of-thumb is that DO MINIMUM network improvements should always be considered where intersections or links are forecast to go overcapacity during the off-peak or adjacent-to-peak time periods.
- For the purposes of benefit/cost analysis, when such improvements are included the DO MINIMUM they should include both the cost of the improvement and the changes in the network compared to the DO NOTHING. In general, the benefits of the DO MINIMUM should far outweigh the cost of the Minimum improvements. If the cost of the DO MINIMUM is significant (approximately 20% or more of the cost of the cheapest DO SOMETHING scenario), the DO MINIMUM itself should be evaluated against a true DO NOTHING. If the expenditure is justifiable in this way, the minor improvements should be included in the DO MINIMUM for comparison with the major DO SOMETHING improvements.
It should be noted that even a literal DO NOTHING base case is normally not a "no change" one. If the project is located in an area where traffic volume is growing, congestion will increase in the future and the road user costs will increase over time.
The DO SOMETHING scenario (also called the or BUILD scenario) is the highway proposal under consideration. Usually there will be more than one feasible DO SOMETHING option. The number and nature of the DO SOMETHING options may change as the planning of the project proceeds. At early stages, a wide range of different options may be considered. At later stages, the range will be narrower but DO SOMETHING options may be refined to highlight more detailed differences such as intersection design or mainline standards.
Where public transportation is considered to be one of the feasible options for solving a particular transportation problem, it is important that all options--road and public transportation alike--are assessed consistently. The modeling and evaluation techniques should match the particular circumstances, taking into account any relevant trade-offs between modes. For example, implementing a signal system that extends the green time for buses traveling along an urban street corridor may improve travel time for bus passengers while imposing a modest travel time penalty on other users of the street (this in turn could lead to some car drivers switching to a different route). Additional rules and guidelines for analysis of public transportation projects can be found on the Federal Transit Administration website.
In microsimulation model development, a project fork occurs when the model developer takes a copy of a source model and begins to modify the network to create a distinct network or scenario. For example, the base-year model and the design-year do-minimum model may represent two forks of the same project. The lineage of a model can be represented as a Model Development Tree.
Forking is a necessary part of the microsimulation modeling process, but serious issues can arise when a model is forked prematurely. When a project has a compressed schedule there may be a temptation to begin development of the design-year DO SOMETHING model before the calibration of the base-year model has been completed. This is counterproductive because all of the changes necessary to achieve proper calibration of the "parent" base-year model will also have to be made in each of the "child" networks. Tracking and finding the changes can be tedious. To identify all of the differences between forks, it is usually necessary to review of each of the text files (or other encoded files) that represent the model’s internal structure; it is almost impossible to find all of the differences just by viewing the model animation. Consequently, premature forking can double or triple the total number of hours necessary to complete the project.
Premature forking can also create model performance inconsistencies that interfere with comparing scenarios. Model reviewers have to spend extra time making sure that the scenarios are consistent with one another, which can distract from making sure the scenarios were modeled effectively. And when clients have to divert their attention to scenario consistency issues, they have less time to assimilate the results and determine how to present them to decision-makers and the public.
To assure consistent performance of all forks, considerable effort that may be necessary to re-trace the steps that were taken to calibrate the base model and replicate them in other forks. In a prematurely-forked model it is often difficult to determine which fork is definitive, since similar calibration problems may have been solved in different ways in each fork. If different individuals have been involved in the development of each fork, this can lead to disagreements among members of the model development team. Therefore, it is always best to complete one fork before starting work on the next. Excessive forking should be avoided. While it is easy to create a fork, it can require considerable effort to continue its development and support. Without adequate resources, forks can quickly become inactive or obsolete or increasing the cost and complexity of the Model Maintenance Plans.
Generally, it is advantageous to minimize the number of forks by using the same network structure for all time periods (AM Peak, PM Peak, Adjacent-to-Peak, Off-Peak, etc.). In most cases it is best to avoid creating separate forks for different time periods and calibrating each time period separately. For example, the EXISTING CONDITIONS (base-year no-build) model should usually consist of a single traffic network with interchangeable Origin-Destination (OD) matrices representing the AM Peak, PM Peak, Adjacent-to-Peak, Off-Peak, etc. Since most networks have a temporal traffic flow pattern (for example, traffic toward downtown heavier in the AM peak and traffic toward the suburbs heavier in the PM peak), typically this can be accomplished as follows:
a. Start the calibration using the AM Peak OD matrix.
b. Calibrate the same network using the PM Peak OD matrix.
c. Re-run the network using the AM Peak matrix to verify that it is still properly calibrated.
The correct sequence of model forking is as follows:
- Develop and calibrate the EXISTING CONDITIONS (base year no-build) model.
- Develop the origin-destination matrix for the design year.
- Try running the design year OD matrix with the no-build network. Usually this DESIGN YEAR DO-NOTHING network will fail, but a copy should be saved in case it is necessary to document the problems that would occur in a true no-build network.
- Using the DESIGN YEAR DO-NOTHING network as a starting point, create the DESIGN YEAR DO-MINIMUM network. Verify calibration.
- Using the DESIGN YEAR DO-MINIMUM network a starting point, create one or more DESIGN YEAR DO-SOMETHING networks. Verify calibration of each network.
|To minimize forking risks, the Project Manager should:|
Typical Microsimulation Deliverables
Task 1: Base Model Development
- Microsimulation software file for the EXISTING CONDITIONS model.
- Model Calibration Report for the EXISTING CONDITIONS model indicating conformance with the Model Calibration Guidelines.
- Nework Audit or Independent Peer Review Report for EXISTING CONDITIONS model (delivered by independent reviewer).
- Video clips of congested/problematic areas in the existing network.
- Report describing the existing network, including screen shots of any existing congested/problematic areas.
Task 2: Development of Future DO NOTHING Model
- Microsimulation software file for design year DO NOTHING model.
- Report describing congested/problematic areas in the DO NOTHING.
Task 3: Development of Future DO MIMIMUM Model
- Microsimulation software file for design year DO MINIMUM model.
- Report describing Minimum improvements required to achieve flowing traffic in the design year DO MINIMUM model.
- If an externally prepared design year forecast is available, a Model Calibration Report for the DO MINIMUM model, indicating conformace with the forecasts.
- If the microsimulation is model is being used as the forecasting tool, a graphical and tabular Summary of Forecasted Growth (increase or decrease in volumes between the EXISTING CONDITIONS and DO MINIMUM models). This report should include both the absolute increase and the percent increase for each link and turn.
- Nework Audit or Independent Peer Review Report for DO MIMIMUM model (delivered by independent reviewer).
- Video clips of congested/problematic areas in the DO MIMIMUM network.
Task 4: Development of Future DO SOMETHING Models
- Microsimulation software files for each design year DO SOMETHING scenario.
- Report describing DO SOMETHING scenario results, including numerical tables and screen shots comparing the DO MINIMUM and DO SOMETHING results.
- Video clips of implemented design solutions in the DO MIMIMUM network.
|The Project Manager should:|