This article is the first part of a two-part series on how to design and build effective mission operations (MissionOps) for space.
The growth of the commercial space industry around the world means that every year there are more missions, run by both new and established teams, that require effective operational management in order to succeed.
In this piece, produced in collaboration with MissionOps software developer Epsilon3, a paying participant in the satsearch membership program, we take a deep dive into how to create an operational structure designed to de-risk your mission and help ensure success.
Please note that this is part one of a two-part article – you can find part two here.
What are space MissionOps?
Space mission operations, often referred to as MissionOps, are the collection of tools, processes, procedures, resources, and approaches that govern how to plan, manage, control, and coordinate the space- and ground-based assets needed to fulfill mission objectives, in specified timelines.
In the diagram below you can see an overview of a typical MissionOps concept. In this setup the Mission Operation Centre (MOC) represents the core coordinating entity responsible for:
- mission timeline planning,
- spacecraft, and payload activity scheduling,
- real-time command and monitoring of the spacecraft,
- flight performance and status assessment,
- in-flight emergency responses, and
- data archiving.
Image source: acsce.edu
An effective MissionOps setup covers activities prior to launch as well as those performed while the asset is in space. Pre-launch mission operation activities will include:
- clarifying mission objectives,
- producing a detailed concept of operations (ConOps),
- developing operator user guides and training resources,
- defining procedure management and update processes and responsibilities,
- hardware and software testing and simulation, and
- other forms of hands-on training.
Launch mission operations will involve the various required testing, qualification, and integration processes required by the launch provider and vehicle (as well as the satellite dispenser, if being used). These are planned out along with the logistics involved in transporting and setting up the assets.
All operational protocols and procedures required to set up the mechanical and electrical ground support equipment, including ground- and space-based payload processing resources, also form part of MissionOps at this stage.
These include communication, documentation, and procedure reviews of all processes and responsibilities. Standard documentation practices involve developing and maintaining a complete and highly usable collection of spacecraft and ground system user guides, both standard and contingency operational plans, training and configuration management plans, work plan schedules, flight rules and operational logs.
This article series will arm you with a range of ideas, considerations, and knowledge needed to develop such a system. But before diving into this guidance, let’s first review a few different examples of missions operations in action, to better understand the concept.
MissionOps examples
Ultimately, every space mission will have some form of MissionOps, no matter what scale it will be operating at, or how many people are involved. For obvious reasons, the larger and more ambitious the mission, the more extensive the MissionOps framework and resources will be.
Here are two examples of large-scale missions with a high-level overview of parts of their operations, illustrating just a few of the many MissionOps concepts that a team needs to cover:
The automated and manual MissionOps of the Hubble Space Telescope
Hubble’s mission operations [PDF] consist of a Mission Operations Room (MOR) and an Operations Support Room (OSR), based at NASA’s Goddard Space Flight Center (GSFC).
Before the implementation of automated operations, a team of operators were required to monitor the console at the MOR around the clock. Today, console operations are staffed only eight hours a day, five days a week.
In case of any anomaly detected by the spacecraft or ground system, the appropriate members of the operations team are alerted immediately through a reliable text messaging system.
If the established anomaly is clear and well-understood, and a specific predetermined response can be identified in the MissionOps framework, the automated system will typically initiate the response and/or alert the operator as needed.
This is part of the system that has enabled the Hubble team to move away from a constant staff presence for monitoring – reducing the burden on the coordination of personnel, status updating, and report logging, as well as the potential for human error.
Human-flight mission operations
Any spacecraft carrying humans has an entirely different level of risk and safety considerations for mission operations compared to autonomous/robotic systems, along with the added complexity of life support, communications, and medical resources on-board.
The best example of this is the famous flying laboratory in space; the International Space Station (ISS). The ISS has hundreds of pages of detailed procedures exclusively drafted for the activities of crew members, in order to ensure their safety and comfort.
In the case of the ISS, there is the added complexity that all MissionOps documents and communication materials have to be accessible using systems in various different countries and in several languages.
Clarity in operations is therefore a fundamental requirement, particularly in an emergency. As an example, the warning message below was received on January 14, 2015, at 02:49 Central Standard Time, along with a red alarm, on the front wall display at ISS Mission Control at Houston, Texas, US:
“TOXIC ATMOSPHERE Node 2 LTL IFHX NH3 Leak Detected.”
The warning message is an example of a mission-critical situation where there is a need for clear procedures to be in place. Such procedures need to be codified and understood by multiple mission operators and should be designed to convey important messages with all necessary (and no superfluous) data so that countermeasures can be implemented in time.
During the assembly missions of ISS, a team of flight controllers worked around the clock to plan for the consecutive missions and support the transport of resources, crew members and their Extravehicular Activity (EVA) operations.
It takes years to plan for such a mission, involving the development of detailed timelines, flight rules and procedures. Once such MissionOps materials are created teams are then trained extensively on handling various mission critical situations.
Other examples of important areas of MissionOps for human spaceflight missions are:
Medical and protective operations – such as clothing monitoring and maintenance, crew health monitoring, routine medical procedures, and specific emergency medical procedures.
Extravehicular Activity (EVA) operations – such as the space walks of astronauts and cosmonauts, real-time support operations for EVA, and the design, components, and operation of the internal spacesuit and additional EVA suit.
In human missions a serious error could quickly become a tragedy, but proper mission planning has, thankfully, limited such instances in space exploration to just a few well-known sad events.
As there have been many more non-crewed space missions, there have been more instances of catastrophic errors. In the next section we review three brief examples of these that further show the importance of high quality MissionOps.
Failed space missions – miscommunication and poor documentation
Operator or engineer miscommunication and a lack of proper documentation have led to a variety of costly mistakes in space mission operations. Here are a few examples of missions that suffered failures due to human error rather than design error.
Vega in 2020
In 2020 the Vega rocket was launched from the Kourou spaceport in French Guiana. Just after eight minutes from launch, the rocket shot off course and eventually crashed, losing the two Earth Observation (EO) satellites on-board.
Investigations uncovered that the cause of the mission failure was inverted cables installed by mistake in the upper stage engine of the Vega rocket. These caused a reverse thrust response to the commands and led to tumbling of the upper stage after liftoff.
A more rigorous installation recording and review process may have led to this error being uncovered prior to launch.
NOAA-N’ in 2003
In 2003 one of the technicians removed 24 bolts holding NASA’s National Oceanic and Atmospheric Administration’s NOAA-N’ EO satellite, but failed to adequately document the operation.
The satellite was later moved by a different team and was accidentally dropped on the floor, causing damage to the system, because of the missing bolts.
Again, proper records of such operations, and an established process for accessing and reviewing them, could have avoided this issue.
Source: NASA (Link)
Mars Climate Orbiter, 1988
In 1988, contact with the NASA Mars Climate Orbiter, designed to study Mars from its orbit, was famously lost and it was concluded that the system disintegrated upon arrival in the atmosphere of Mars.
The failure was found to have been caused by a miscommunication in the use of units in the navigation system. One of the teams used the metric system while the other used the imperial system, and this discrepancy had not been effectively communicated.
An accurate conversion between these units was therefore not made and the mission suffered a catastrophic error as a result.
These examples show how seemingly small errors can occur, leading to major failures, even in missions with a large number of experienced personnel, redundancies, established processes, and failsafes.
Next we take a closer look at how such issues in mission operations can occur.
Challenges in current procedures
There is a lot more depth and diversity to the challenges that occur in mission operations than simple miscommunication and poor documentation of course.
Operational procedures used in modern space missions face a variety of issues due to complexities of the technologies, processes, and application areas, as well as the enormously demanding environment of space. Here is a list of some of the common challenges:
- A lack of efficient software tools for operating vehicles.
- A lack of standardization to facilitate agile and lean methods of working.
- Complexity due to operations handled by multiple stakeholders in multiple locations, across various languages, timezones, systems, and regulatory areas.
- A lack of effective version control, particularly regarding the interrogation of different iterations.
- Ensuring interoperability across the multiple tools and file formats that are typically used for different stages of the operation such as: Excel, paper, Word, Confluence, OneNote, Jira, emails, and PDFs.
- The inability to easily track, or be notified of, sign-offs by different operators.
- The difficulties in figuring out the history of events due to data spread across multiple sources. For example, if a new test is performed and the results are unexpectedly different from previous analyses, it can be difficult to search through historic communications to find out previous testing setups to determine what caused the difference.
- It is hard to track version control when printed and digital documents are used together, or alternately.
- The lack of interoperability between some software tools and platforms, particularly during testing, and operational procedures that rely on in-house programs.
These issues essentially result from inefficiencies, a lack of resources, or unsuitable processes relating to people, systems, and approaches to using them. Relying heavily on older tools in particular can lead to a high probability of errors, missing details, and miscommunications when data is moved around and iterated on.
Personnel management is also an important aspect of MissionOps. There is a growing need for faster and more seamless on-boarding of new staff in the modern industry. Today’s missions and services are placing increasing demands on teams, and specialists are hard to replace.
Mission teams are also increasingly using automation for various operating and reporting procedures, which can certainly enhance efficiency. However teams still need to understand those processes that are automated and will have to perform some manual actions.
In every operation there is a balance between automation and manual activities – and both need to be mapped and understood by the team in order for them to be optimized.
The documentation, training, and processes put into place should be designed to enable new people to slot into operations as seamlessly as possible.
Such staff turnover is usually more of an issue for smaller teams. In bigger teams, it is more likely that there will be other people familiar with the operations and who have the expertise to replace an operator, even if only temporarily. But small mission teams and companies will likely need an outside hire, meaning that good documents and processes are vital.
As you can see, space mission operations are both critically important and complex, time-consuming processes. In the second part of this guide we take a closer look at documentation, electronic communication and protocol management, and operator training, in order to provide advice on the development of effective MissionOps.
In the meantime, to find out more about Epsilon3’s work please view their supplier hub on satsearch here.