Launch-eve evidence room · record frozen at close of

Open on October 1? The public record holds promises and plans, not a proven pass.

A readiness checklist for the accountable official, built only from four dated public excerpts. Each line separates what was promised, what was planned, what was observed, what an independent party verified, and what is unknown.

The answer

Is consumer enrollment demonstrably ready on the public record? Not demonstrably ready.

  • 0 independently verified passes appear in the public excerpts for any of the 7 readiness requirements below.
  • 10 planned milestones fell due on or before 2013-09-30 (month-only dates read as due at month end, the lenient reading). None has an observed completion in the excerpts; 2 have only a CMS self-reported completion claim.
  • The latest independent assessment, HC1 GAO-13-601 published , is 103 days old and rests mostly on records through early May. It said smooth implementation by October 2013 cannot yet be determined.
  • Eligibility and enrollment testing was scheduled to finish by , the opening day itself, according to HC1 GAO-13-601 (p. 21). No capacity or response-time test appears in any excerpt.

Demand before opening

  1. R1 A dated go/no-go record listing each criterion in this table as pass, fail or waived, with the accountable signer.
  2. R1 An independent readiness assessment more recent than the GAO report of 2013-06-19.
  3. R2 End-to-end test report covering account, application, eligibility determination, plan selection and enrollment record, with pass and fail counts and open severe defects, signed by a tester independent of the build team.
  4. R3 A partner-by-partner and state-by-state list of external test completion dates and results, and the hub operational readiness sign-off.
  5. R4 Count of certified QHPs per FFE state, the load confirmation date, and a reconciliation of displayed rates and benefits against issuer filings.
  6. R5 Navigator award list with award dates and training completion; call center staffing level, scripts and test-call results for applications; in-person system availability test.
  7. R6 Load test results stating tested concurrent users and transactions against forecast demand, measured response times against SLA targets, and a tested queue, throttle and rollback capability.
  8. R6 Signed SLAs with states and federal partner agencies.
  9. R7 Current contract and task-order list with obligations to date, scope of work, and the party accountable for end-to-end integration testing.

If that evidence cannot be produced: a narrower opening

  1. Keep the public date but open only functions that have a dated, independently checked pass; label the rest as not yet available.
  2. Admit traffic at no more than the tested capacity behind a queue, with a published status page and a tested rollback.
  3. Hold plan selection and enrollment transmission to issuers until the end-to-end test passes; let consumers create accounts, browse plans and start applications, with the call center and in-person channels as documented fallbacks.
  4. Publish a daily readiness ledger using the same five statuses and expand scope only when a criterion moves to an independently verified pass.
  5. Use the 92 days between opening and the first day of coverage as a ceiling for staged expansion, not a guarantee: the excerpts do not state the enrollment cutoff for January 1 coverage. (HC2 CMS release, HC4 House majority memo)

How to read every line: five statuses

Status marks are shapes and words, not colors. A planned date that has passed stays planned until the record shows otherwise.

  • Claim a statement by an agency, contractor or committee staff about what is or will be true; not a test result.
  • Planned a date by which something is expected; a passed date is not evidence of completion.
  • Observed an event the record reports as having happened (an award, a start, a publication, an obligation).
  • Verified pass a test or review by a party other than the builder showing the function works. None appears in these excerpts.
  • Unknown the public excerpts do not show whether it happened.

Launch-eve requirements and readiness

One row per readiness question, linking the public promise to eligibility, data exchange, plans, consumer assistance, capacity and contractor scope. Day counts appear only between exact dates; "elapsed" means days from a planned date to review close on 2013-09-30.

Requirements table, record as of close of 2013-09-30
Readiness questionPromised (claim)Planned deadlineObserved in recordIndependently verified passCannot be proved from excerptsDemand before opening
R1 Overall go-liveIs there a current, independent determination that the marketplace can open for consumer enrollment?Not demonstratedInspect the quotes for R1 Open enrollment begins Oct. 1, 2013; HHS says it is on target and, per GAO, expressed confidence. HC2 CMS release, HC1 GAO-13-601 (Highlights) None dated in excerpts.
  • Observed early May (approximate precision): GAO findings mostly rest on CMS records through early May (approximate) HC1 GAO-13-601
  • Observed : GAO report published: timely, smooth implementation cannot yet be determined HC1 GAO-13-601
  • Unknown GAO: whether implementation will be timely and smooth cannot yet be determined HC1 GAO-13-601 (Highlights)
None in public excerpts
  • No independent readiness determination dated after 2013-06-19 appears in the excerpts; GAO then said it cannot yet be determined.
  • The excerpt saying work was delayed by about 2 months does not name what was delayed.
  • Whether the majority staff concerns are borne out: the memo excerpt cites no test result.
  • A dated go/no-go record listing each criterion in this table as pass, fail or waived, with the accountable signer.
  • An independent readiness assessment more recent than the GAO report of 2013-06-19.
R2 Eligibility and enrollmentCan a consumer create an account, apply, receive an eligibility determination and enroll?Not demonstratedInspect the quotes for R2 By October, consumers will be able to create accounts, complete an online application, and shop. HC2 CMS release
  • Planned : Eligibility and enrollment IT developed and tested; enrollment begins HC1 GAO-13-601
Nothing observed in excerpts. None in public excerpts
  • Whether development and testing finished: GAO reported it as an expectation for October 1, 2013, the opening day itself.
  • Whether the full path from account to enrollment record has passed any end-to-end test.
  • End-to-end test report covering account, application, eligibility determination, plan selection and enrollment record, with pass and fail counts and open severe defects, signed by a tester independent of the build team.
R3 Data exchange (data hub)Do verification data exchanges with federal partners and states work in production conditions?Not demonstratedInspect the quotes for R3 No data hub promise appears in the public release; the online application promise depends on it. HC2 CMS release
  • Planned July 2013 (month precision): Data hub required functionality developed; no day count HC1 GAO-13-601
  • Planned September 2013 (month precision): External testing with all federal partners and all states complete (by the beginning of the month); no day count HC1 GAO-13-601
  • Planned September 2013 (month precision): Data hub full implementation and operational readiness; no day count HC1 GAO-13-601
  • Observed October 2012 (month precision): Data hub internal and external testing began, as planned (a start, not a completion) HC1 GAO-13-601
  • Observed As of GAO reporting, final testing with federal and state partners remained to be completed HC1 GAO-13-601 (Highlights)
None in public excerpts
  • Whether external testing with all federal partner agencies and all states finished.
  • Whether the hub reached full implementation and operational readiness in September 2013.
  • A partner-by-partner and state-by-state list of external test completion dates and results, and the hub operational readiness sign-off.
R4 Plans (QHPs)Are certified plans loaded with correct information so consumers can compare and shop?Not demonstratedInspect the quotes for R4 Consumers will be able to shop for and compare qualified health plans in October. HC2 CMS release
  • Planned : Health plans evaluated and certified as QHPs; elapsed 61 days by review close, no completion shown HC1 GAO-13-601
  • Planned : QHP information finalized and loaded into the exchange website; elapsed 15 days by review close, no completion shown HC1 GAO-13-601
Nothing observed in excerpts. None in public excerpts
  • How many plans were certified, and whether certification finished by July 31, 2013.
  • Whether plan information was finalized and loaded by September 15, 2013, and checked against issuer submissions.
  • Count of certified QHPs per FFE state, the load confirmation date, and a reconciliation of displayed rates and benefits against issuer filings.
R5 Consumer assistanceWill navigators, the call center and in-person help be able to assist applicants on day one?Not demonstratedInspect the quotes for R5 Beginning Oct. 1, 2013, the call center will assist consumers with application completion and plan selection. HC2 CMS release
  • Planned June 2013 (month precision): Call center to begin operations; no day count; CMS self-reported completion on 2013-06-24, not verified HC1 GAO-13-601
  • Planned June 2013 (month precision): HealthCare.gov relaunch with enrollment focus; no day count; CMS self-reported completion on 2013-06-24, not verified HC1 GAO-13-601
  • Planned : Navigator awards anticipated (single round) HC1 GAO-13-601
  • Planned October 2013 (month precision): In-person assistance system for FFE states available; no day count HC1 GAO-13-601
  • Observed : Navigator funding announcement issued after a delay HC1 GAO-13-601
  • Observed : Call center operations contract awarded HC1 GAO-13-601
None in public excerpts
  • Whether navigator awards were made on the anticipated date and whether training finished; that date left 47 days before opening.
  • Whether the call center is staffed, scripted and tested for application assistance, not only education.
  • Whether the in-person assistance system for FFE states is available.
  • Navigator award list with award dates and training completion; call center staffing level, scripts and test-call results for applications; in-person system availability test.
R6 Capacity and performanceCan the system carry expected opening-day volume within agreed response times?Not demonstratedInspect the quotes for R6 No public capacity or response-time promise appears in any excerpt.
  • Planned July 2013 (month precision): SLAs with states (response time, availability) completed; no day count HC1 GAO-13-601
  • Planned July 2013 (month precision): SLAs with federal partner agencies completed; no day count HC1 GAO-13-601
Nothing observed in excerpts. None in public excerpts
  • No load, volume or response-time test result appears in any excerpt.
  • Expected peak demand, tested concurrent users and SLA targets are not in the excerpts.
  • Whether the state and federal partner SLAs planned for July 2013 were signed.
  • Load test results stating tested concurrent users and transactions against forecast demand, measured response times against SLA targets, and a tested queue, throttle and rollback capability.
  • Signed SLAs with states and federal partner agencies.
R7 Contractor scope and moneyWho is building what, under which contract, with how much obligated?Not demonstratedInspect the quotes for R7 The contractor announced it will develop and implement the Federal Exchange. HC3 CGI release None dated in excerpts.
  • Observed : CMS obligations to CGI Federal counted through this date HC1 GAO-13-601
  • Observed Obligated to CGI Federal through : $87,997,938 HC1 GAO-13-601 (p. 33)
  • Claim Award estimate US$93.7 million, base awarded US$55.7 million: contractor figures, not federal spending HC3 CGI release
None in public excerpts
  • Whether the GAO-listed CGI Federal contract is the same award CGI announced on 2011-12-01: the announcement gives no task-order identifier.
  • Obligations after March 31, 2013 and any contract modifications.
  • Which party is accountable for end-to-end integration and testing across contractors and partners.
  • Current contract and task-order list with obligations to date, scope of work, and the party accountable for end-to-end integration testing.

Evidence room: inspect the quotes behind each question

Each panel holds the verbatim lines from the excerpt files, with the status of each line and what it does not prove. Open any panel directly; with scripts on, the chooser below opens one question at a time.

R1 Overall go-live: 9 quoted lines

Is there a current, independent determination that the marketplace can open for consumer enrollment?

"HHS is on target for open enrollment in the Marketplace, which begins Oct. 1, 2013, and other key milestones approaching in the months ahead. Coverage will begin Jan. 1, 2014."
Claim agency public promise · HC2 CMS release · excerpt line 8
"The new tools will help Americans understand their choices and select the coverage that best suits their needs when open enrollment in the new Health Insurance Marketplace begins October 1."
Claim agency public promise · HC2 CMS release · excerpt line 4
"In commenting on a draft of this report, the Department of Health and Human Services emphasized the progress it has made in establishing exchanges, and expressed its confidence that exchanges will be open and functioning in every state by October 1, 2013."
Claim agency claim reported by GAO · HC1 GAO-13-601 (Highlights) · excerpt line 9
"CMS completed many activities necessary to establish FFEs by October 1, 2013, although many remain to be completed and some were behind schedule."
Observed independent audit finding · HC1 GAO-13-601 (Highlights) · excerpt line 5
"While CMS has met project schedules, several critical tasks, such as final testing with federal and state partners, remain to be completed."
Observed independent audit finding · HC1 GAO-13-601 (Highlights) · excerpt line 6
"Whether these efforts will assure the timely and smooth implementation of the exchanges by October 2013 cannot yet be determined."
Unknown independent audit finding · HC1 GAO-13-601 (Highlights) · excerpt line 8
"have been delayed by about 2 months, which has delayed training and other activities."
Observed independent audit finding (subject not in excerpt) · HC1 GAO-13-601 (Highlights) · excerpt line 7
"Open enrollment in the PPACA exchanges begins October 1, 2013, with coverage effective January 1, 2014."
Claim schedule restated by majority staff · HC4 House majority memo · excerpt line 5
"As October 1, 2013, approaches, many issues related to readiness, testing, and functionality of exchanges remain. Missed deadlines, delays, and untimely guidance likely will affect areas such as consumer assistance and experience, eligibility accuracy, integration with existing State programs, and interagency coordination."
Claim majority staff concern, not a test · HC4 House majority memo · excerpt line 6

Not shown by these lines: No independent readiness determination dated after 2013-06-19 appears in the excerpts; GAO then said it cannot yet be determined. The excerpt saying work was delayed by about 2 months does not name what was delayed. Whether the majority staff concerns are borne out: the memo excerpt cites no test result.

R2 Eligibility and enrollment: 4 quoted lines

Can a consumer create an account, apply, receive an eligibility determination and enroll?

"The website will add functionality over the summer so that, by October, consumers will be able to create accounts, complete an online application, and shop for qualified health plans."
Claim agency public promise · HC2 CMS release · excerpt line 6
"The re-launched Healthcare.gov and new call center will help consumers prepare for the new coverage opportunities coming later this year," said Centers for Medicare & Medicaid Services Administrator Marilyn Tavenner. "In October, HealthCare.gov will be the online destination for consumers to compare and enroll in affordable, qualified health plans."
Claim agency public promise, direct quotation · HC2 CMS release · excerpt line 5
"CMS expects to complete development and testing of the information technology systems necessary for FFEs to determine eligibility for enrollment into a QHP and to enroll individuals by October 1, 2013, when enrollment is scheduled to begin for the 2014 plan year."
Planned agency plan reported by GAO · HC1 GAO-13-601 (p. 21) · excerpt line 12
"While CMS has met project schedules, several critical tasks, such as final testing with federal and state partners, remain to be completed."
Observed independent audit finding · HC1 GAO-13-601 (Highlights) · excerpt line 6

Not shown by these lines: Whether development and testing finished: GAO reported it as an expectation for October 1, 2013, the opening day itself. Whether the full path from account to enrollment record has passed any end-to-end test.

R3 Data exchange (data hub): 5 quoted lines

Do verification data exchanges with federal partners and states work in production conditions?

"According to program officials, CMS established milestones for completing the development of required data hub functionality by July 2013, and for full implementation and operational readiness by September 2013."
Planned agency plan reported by GAO · HC1 GAO-13-601 (pp. 22-24) · excerpt line 15
"CMS began conducting both internal and external testing for the data hub in October 2012, as planned."
Observed event reported by GAO · HC1 GAO-13-601 (pp. 22-24) · excerpt line 16
"Completion of external testing with all federal partner agencies and all states is to be completed by the beginning of September 2013."
Planned agency plan reported by GAO · HC1 GAO-13-601 (pp. 22-24) · excerpt line 19
"While CMS has met project schedules, several critical tasks, such as final testing with federal and state partners, remain to be completed."
Observed independent audit finding · HC1 GAO-13-601 (Highlights) · excerpt line 6
"SLAs between CMS and its federal partner agencies that provide verification data are expected to be completed in July 2013; and"
Planned agency plan reported by GAO · HC1 GAO-13-601 (pp. 22-24) · excerpt line 18

Not shown by these lines: Whether external testing with all federal partner agencies and all states finished. Whether the hub reached full implementation and operational readiness in September 2013.

R4 Plans (QHPs): 4 quoted lines

Are certified plans loaded with correct information so consumers can compare and shop?

"Once received, CMS, with the assistance of its contractor, expects to evaluate and certify health plans as QHPs by July 31, 2013."
Planned agency plan reported by GAO · HC1 GAO-13-601 (pp. 24-25) · excerpt line 22
"CMS then expects to finalize the QHP information and load it into the exchange website by September 15, 2013."
Planned agency plan reported by GAO · HC1 GAO-13-601 (pp. 24-25) · excerpt line 23
"The website will add functionality over the summer so that, by October, consumers will be able to create accounts, complete an online application, and shop for qualified health plans."
Claim agency public promise · HC2 CMS release · excerpt line 6
"The re-launched Healthcare.gov and new call center will help consumers prepare for the new coverage opportunities coming later this year," said Centers for Medicare & Medicaid Services Administrator Marilyn Tavenner. "In October, HealthCare.gov will be the online destination for consumers to compare and enroll in affordable, qualified health plans."
Claim agency public promise, direct quotation · HC2 CMS release · excerpt line 5

Not shown by these lines: How many plans were certified, and whether certification finished by July 31, 2013. Whether plan information was finalized and loaded by September 15, 2013, and checked against issuer submissions.

R5 Consumer assistance: 6 quoted lines

Will navigators, the call center and in-person help be able to assist applicants on day one?

"Between now and the start of open enrollment, the Marketplace call center will provide educational information and, beginning Oct. 1, 2013, will assist consumers with application completion and plan selection."
Claim agency public promise · HC2 CMS release · excerpt line 7
"The re-launched Healthcare.gov and new call center will help consumers prepare for the new coverage opportunities coming later this year," said Centers for Medicare & Medicaid Services Administrator Marilyn Tavenner. "In October, HealthCare.gov will be the online destination for consumers to compare and enroll in affordable, qualified health plans."
Claim agency public promise, direct quotation · HC2 CMS release · excerpt line 5
"However, the announcement was delayed until April 9, 2013, and CMS officials indicated that there would be one round of awards, with an anticipated award date of August 15, 2013."
Mixed event and plan reported by GAO · HC1 GAO-13-601 (p. 26) · excerpt line 26
"CMS awarded the contract for operation of the call center on April 15, 2013, with the call center to begin operations by June 2013."
Mixed event and plan reported by GAO from CMS · HC1 GAO-13-601 (p. 29) · excerpt line 29
"CMS planned to relaunch the healthcare.gov website in June 2013 to provide a greater focus on exchange enrollment."
Planned agency plan reported by GAO from CMS · HC1 GAO-13-601 (p. 29) · excerpt line 30
"CMS plans that the system used by the FFE in states not assisting with consumer assistance activities will be available by the October 2013 initial enrollment date."
Planned agency plan reported by GAO from CMS · HC1 GAO-13-601 (p. 29) · excerpt line 31

Not shown by these lines: Whether navigator awards were made on the anticipated date and whether training finished; that date left 47 days before opening. Whether the call center is staffed, scripted and tested for application assistance, not only education. Whether the in-person assistance system for FFE states is available.

R6 Capacity and performance: 2 quoted lines

Can the system carry expected opening-day volume within agreed response times?

"Service Level Agreements (SLA) between CMS and the states, which define characteristics of the system once it is operational, such as transaction response time and days and hours of availability, are planned to be completed in July 2013;"
Planned agency plan reported by GAO · HC1 GAO-13-601 (pp. 22-24) · excerpt line 17
"SLAs between CMS and its federal partner agencies that provide verification data are expected to be completed in July 2013; and"
Planned agency plan reported by GAO · HC1 GAO-13-601 (pp. 22-24) · excerpt line 18

Not shown by these lines: No load, volume or response-time test result appears in any excerpt. Expected peak demand, tested concurrent users and SLA targets are not in the excerpts. Whether the state and federal partner SLAs planned for July 2013 were signed.

R7 Contractor scope and money: 4 quoted lines

Who is building what, under which contract, with how much obligated?

"The Centers for Medicare & Medicaid Services (CMS) Center for Consumer Information and Insurance Oversight (CCIIO) has awarded CGI Federal Inc. (CGI), a wholly-owned U.S. operating subsidiary of CGI Group Inc. (NYSE: GIB) (TSX: GIB.A), a contract to build the U.S. government’s federally-facilitated health insurance exchange, Federal Exchange (FX). The contract has an estimated total contract value of US$93.7 million over a two-year base with three, one-year option periods. The US$55.7 million base contract has been awarded."
Claim contractor award announcement · HC3 CGI release · excerpt line 4
"Under the contract, CGI will work with CCIIO to develop and implement the Federal Exchange in line with the provision in the Affordable Care Act that requires the creation of a competitive health insurance marketplace for States and territories to help bring insurance coverage to 34 million additional Americans."
Claim contractor award announcement · HC3 CGI release · excerpt line 5
"For one contract, with CGI Federal, CMS obligated about $88 million for activities to support establishment of the FFEs, such as information technology and technical assistance."
Observed obligation reported by GAO · HC1 GAO-13-601 (p. 32) · excerpt line 34
"CGI Federal Inc" | "FFE information technology and healthcare.gov" | "$87,997,938"
Observed obligation table row reported by GAO · HC1 GAO-13-601 (p. 33) · excerpt line 36

Not shown by these lines: Whether the GAO-listed CGI Federal contract is the same award CGI announced on 2011-12-01: the announcement gives no task-order identifier. Obligations after March 31, 2013 and any contract modifications. Which party is accountable for end-to-end integration and testing across contractors and partners.

Dated timeline on a true date axis

Vertical position is proportional to calendar date in each panel. Exact dates are point marks; month-only dates are bars spanning the whole month, because the record does not say which day. Labels are pushed apart where dates crowd, and a thin leader line points to the true date.

Panel A: award to early testing, Nov 2011 to Dec 2012

Scale: 1 month ≈ 18 units of height.
Panel A timeline, November 2011 to December 2012Contractor award announcement on 2011-12-01 (claim) and data hub testing start in October 2012 (observed, month only). Nov 2011 Dec Jan 2012 Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec HC3 2011-12-01 · Claim CGI announces award: estimated total value and base contract awarded HC1 October 2012 (month) · Observed Data hub internal and external testing began, as planned (a start, not a completion)

Panel B: launch year, Mar 2013 to Jan 2014

Scale: 1 month ≈ 97 units of height. The accent line marks review close.
Panel B timeline, March 2013 to January 2014Readiness milestones by status; every mark has a matching row in the table beside it. Mar 2013 Apr May Jun Jul Aug Sep Oct Nov Dec Jan 2014 HC1 2013-03-31 · Observed CMS obligations to CGI Federal counted through this date HC1 2013-04-09 · Observed Navigator funding announcement issued after a delay HC1 2013-04-15 · Observed Call center operations contract awarded HC1 early May (approx.) · Observed GAO findings mostly rest on CMS records through early May (approximate) HC1 June 2013 (month) · Planned Call center to begin operations HC1 June 2013 (month) · Planned HealthCare.gov relaunch with enrollment focus HC1 2013-06-19 · Observed GAO report published: timely, smooth implementation cannot yet be determined HC2 2013-06-24 · Claim CMS release: site re-launched, new call center, HHS on target HC1 July 2013 (month) · Planned Data hub required functionality developed HC1 July 2013 (month) · Planned SLAs with states (response time, availability) completed HC1 July 2013 (month) · Planned SLAs with federal partner agencies completed HC1 2013-07-31 · Planned Health plans evaluated and certified as QHPs HC1 2013-08-15 · Planned Navigator awards anticipated (single round) HC1 September 2013 (month) · Planned External testing with all federal partners and all states complete (by the beginning of the month) HC1 September 2013 (month) · Planned Data hub full implementation and operational readiness HC4 2013-09-06 · Claim House majority staff memo: readiness issues remain (a concern, not a test) HC1 2013-09-15 · Planned QHP information finalized and loaded into the exchange website 2013-09-30 Review close: frame of this audit, not a source event HC1 2013-10-01 · Planned Eligibility and enrollment IT developed and tested; enrollment begins HC2 2013-10-01 · Claim Call center begins assisting with applications and plan selection HC1 October 2013 (month) · Planned In-person assistance system for FFE states available HC2 2014-01-01 · Claim Coverage begins
Every timeline mark as a plain row (unit: calendar date)
Date (unit: calendar date)StatusEventSource
ClaimCGI announces award: estimated total value and base contract awardedHC3 CGI release
October 2012 (month precision)ObservedData hub internal and external testing began, as planned (a start, not a completion)HC1 GAO-13-601
ObservedCMS obligations to CGI Federal counted through this dateHC1 GAO-13-601
ObservedNavigator funding announcement issued after a delayHC1 GAO-13-601
ObservedCall center operations contract awardedHC1 GAO-13-601
early May (approximate precision)ObservedGAO findings mostly rest on CMS records through early May (approximate)HC1 GAO-13-601
June 2013 (month precision)PlannedCall center to begin operationsHC1 GAO-13-601
June 2013 (month precision)PlannedHealthCare.gov relaunch with enrollment focusHC1 GAO-13-601
ObservedGAO report published: timely, smooth implementation cannot yet be determinedHC1 GAO-13-601
ClaimCMS release: site re-launched, new call center, HHS on targetHC2 CMS release
July 2013 (month precision)PlannedData hub required functionality developedHC1 GAO-13-601
July 2013 (month precision)PlannedSLAs with states (response time, availability) completedHC1 GAO-13-601
July 2013 (month precision)PlannedSLAs with federal partner agencies completedHC1 GAO-13-601
PlannedHealth plans evaluated and certified as QHPsHC1 GAO-13-601
PlannedNavigator awards anticipated (single round)HC1 GAO-13-601
September 2013 (month precision)PlannedExternal testing with all federal partners and all states complete (by the beginning of the month)HC1 GAO-13-601
September 2013 (month precision)PlannedData hub full implementation and operational readinessHC1 GAO-13-601
ClaimHouse majority staff memo: readiness issues remain (a concern, not a test)HC4 House majority memo
PlannedQHP information finalized and loaded into the exchange websiteHC1 GAO-13-601
PlannedEligibility and enrollment IT developed and tested; enrollment beginsHC1 GAO-13-601
ClaimCall center begins assisting with applications and plan selectionHC2 CMS release
October 2013 (month precision)PlannedIn-person assistance system for FFE states availableHC1 GAO-13-601
ClaimCoverage beginsHC2 CMS release

Exact day gaps

Computed only where both ends are exact calendar dates: later date minus earlier date, in calendar days.

Day gaps between exact dates
IntervalFromToDaysSources
CGI award announcement to the date through which GAO counted obligations CGI announces award: estimated total value and base contract awarded CMS obligations to CGI Federal counted through this date486HC3 CGI release, HC1 GAO-13-601
Delayed navigator announcement to anticipated navigator award Navigator funding announcement issued after a delay Navigator awards anticipated (single round)128HC1 GAO-13-601
Anticipated navigator award to opening: the most time awardees could have for training Navigator awards anticipated (single round) Open enrollment begins47HC1 GAO-13-601, HC2 CMS release, HC4 House majority memo
Call center contract award to opening Call center operations contract awarded Open enrollment begins169HC1 GAO-13-601, HC2 CMS release, HC4 House majority memo
Planned QHP certification date to review close: elapsed with no completion evidence in excerpts Health plans evaluated and certified as QHPs Review close (frame of this audit)61HC1 GAO-13-601 and the review frame
Planned QHP data load to review close: elapsed with no completion evidence in excerpts QHP information finalized and loaded into the exchange website Review close (frame of this audit)15HC1 GAO-13-601 and the review frame
Age of the latest independent assessment (GAO) at review close GAO report published: timely, smooth implementation cannot yet be determined Review close (frame of this audit)103HC1 GAO-13-601 and the review frame
CMS 'on target' release to opening CMS release: site re-launched, new call center, HHS on target Open enrollment begins99HC2 CMS release, HC1 GAO-13-601, HC4 House majority memo
House majority staff memo to opening House majority staff memo: readiness issues remain (a concern, not a test) Open enrollment begins25HC4 House majority memo, HC1 GAO-13-601, HC2 CMS release
Opening to first day of coverage Open enrollment begins Coverage begins92HC1 GAO-13-601, HC2 CMS release, HC4 House majority memo
Review close to opening Review close (frame of this audit) Open enrollment begins1HC1 GAO-13-601, HC2 CMS release, HC4 House majority memo and the review frame

Vague dates: no day count computed

  • "October 2012" (data hub testing began): month only. HC1 GAO-13-601 (pp. 22-24)
  • "early May" (GAO evidence cutoff): approximate; from the SOURCES.csv scope note, not a quoted line. HC1 GAO-13-601
  • "June 2013" (call center to begin operations): month only. HC1 GAO-13-601 (p. 29)
  • "June 2013" (HealthCare.gov relaunch): month only. HC1 GAO-13-601 (p. 29)
  • "July 2013" (data hub functionality complete): month only. HC1 GAO-13-601 (pp. 22-24)
  • "July 2013" (SLAs with states): month only. HC1 GAO-13-601 (pp. 22-24)
  • "July 2013" (SLAs with federal partner agencies): month only. HC1 GAO-13-601 (pp. 22-24)
  • "beginning of September 2013" (external testing with all partners complete): vague part of a month. HC1 GAO-13-601 (pp. 22-24)
  • "September 2013" (data hub operational readiness): month only. HC1 GAO-13-601 (pp. 22-24)
  • "October 2013" (in-person assistance system available): month only. HC1 GAO-13-601 (p. 29)
  • "over the summer" (website adds functionality): season, no date. HC2 CMS release
  • "by October" (accounts, application and shopping available): month, no year in clause. HC2 CMS release
  • "In October" (HealthCare.gov as enrollment destination): month, no year in clause. HC2 CMS release
  • "about 2 months" (a delay): approximate duration; the excerpt does not name what was delayed. HC1 GAO-13-601 (Highlights)

Contractor scope: award estimate is not obligation, and obligation is not spending

CGI figures on one scale (unit: US$ million)

Dashed outline: estimate. Open outline: announced base. Solid: obligation.
CGI contract figures in US dollars, by kind Three horizontal bars on a shared scale from 0 to 100 million US dollars. The award estimate and base award come from the contractor announcement; the obligation comes from GAO. None is federal spending. 0 25 50 75 100 US$ million Award estimate, all options (not obligated, not spent) US$93.7 million HC3 CGI release Base contract awarded, as announced by contractor US$55.7 million HC3 CGI release Obligated by CMS through 2013-03-31 (not an outlay) $87,997,938 HC1 GAO-13-601
Contract values as plain rows (unit: US dollars)
FigureAmount (US$)What it isSource
Award estimate, announced US$93.7 millionContractor's estimate of total value if all option periods are exercised; a potential ceiling, not money obligated or spent. Terms: a two-year base with three, one-year option periods.HC3 CGI release
Base contract, announced as awardedUS$55.7 millionContractor's statement that the base contract was awarded; not an audited obligation figure.HC3 CGI release
Obligated by CMS through $87,997,938 (about $88 million in GAO text)Amount CMS obligated to CGI Federal through March 31, 2013 per GAO Table 4; an obligation is a commitment to pay, not an outlay. Line item: FFE information technology and healthcare.gov.HC1 GAO-13-601 (pp. 32-33)
Obligation minus announced base$32,297,938Arithmetic only. The obligation exceeds the announced base by this amount; the excerpts do not say whether options, modifications or another vehicle explain it, or whether GAO's contract is the 2011 award.HC1 GAO-13-601, HC3 CGI release

No ratio of obligations to the award estimate is reported: the two figures come from different publishers, dates and possibly different contract vehicles. The announcement and the obligation count are 486 days apart. Federal spending (outlays) for this contract does not appear in any excerpt.

Majority staff concerns are not verified tests

"As October 1, 2013, approaches, many issues related to readiness, testing, and functionality of exchanges remain. Missed deadlines, delays, and untimely guidance likely will affect areas such as consumer assistance and experience, eligibility accuracy, integration with existing State programs, and interagency coordination."
Claim majority staff concern, not a test · HC4 House majority memo, dated

Majority staff position stated in a hearing memorandum; not a GAO finding and not a measured test result. Test results cited in the excerpt: 0. Treat as a list of questions to ask, neither as proof of failure nor as something the agency must rebut with assertions.

By contrast, GAO is independent but older: its excerpts report plans and partial progress, and say whether implementation will be timely and smooth cannot yet be determined HC1 GAO-13-601 (Highlights).

The strongest case for opening on the promised date

  1. Claim The date is a public commitment restated across the record: CMS and the House majority staff memo raising readiness concerns both give October 1, 2013. HC2 CMS release, HC4 House majority memo
  2. Observed GAO, the only independent source here, found CMS had met project schedules and completed many activities, and that data hub testing began in October 2012 as planned. HC1 GAO-13-601 (Highlights), HC1 GAO-13-601 (pp. 22-24)
  3. Observed Key procurements are observed, not merely planned: the call center contract was awarded on 2013-04-15 and CMS had obligated $87,997,938 to CGI Federal through 2013-03-31. HC1 GAO-13-601 (p. 29), HC1 GAO-13-601 (p. 33)
  4. Claim HHS expressed confidence to GAO, and on 2013-06-24 CMS said HHS is on target and that the site was re-launched with a new call center. HC1 GAO-13-601 (Highlights), HC2 CMS release
  5. Claim Opening enrollment is not the start of coverage: 92 days separate them, leaving time to find and fix problems before anyone's coverage depends on them [INFERENCE from the two dates]. HC2 CMS release, HC4 House majority memo

Limit: Every point above is a claim, a plan or an early-stage observation. None shows that a consumer can complete enrollment end to end at expected volume.

"The re-launched Healthcare.gov and new call center will help consumers prepare for the new coverage opportunities coming later this year," said Centers for Medicare & Medicaid Services Administrator Marilyn Tavenner. "In October, HealthCare.gov will be the online destination for consumers to compare and enroll in affordable, qualified health plans."
Claim Direct quotation in the dated CMS release of · HC2 CMS release

What the public excerpts cannot prove

  • Unknown R1 No independent readiness determination dated after 2013-06-19 appears in the excerpts; GAO then said it cannot yet be determined.
  • Unknown R1 The excerpt saying work was delayed by about 2 months does not name what was delayed.
  • Unknown R1 Whether the majority staff concerns are borne out: the memo excerpt cites no test result.
  • Unknown R2 Whether development and testing finished: GAO reported it as an expectation for October 1, 2013, the opening day itself.
  • Unknown R2 Whether the full path from account to enrollment record has passed any end-to-end test.
  • Unknown R3 Whether external testing with all federal partner agencies and all states finished.
  • Unknown R3 Whether the hub reached full implementation and operational readiness in September 2013.
  • Unknown R4 How many plans were certified, and whether certification finished by July 31, 2013.
  • Unknown R4 Whether plan information was finalized and loaded by September 15, 2013, and checked against issuer submissions.
  • Unknown R5 Whether navigator awards were made on the anticipated date and whether training finished; that date left 47 days before opening.
  • Unknown R5 Whether the call center is staffed, scripted and tested for application assistance, not only education.
  • Unknown R5 Whether the in-person assistance system for FFE states is available.
  • Unknown R6 No load, volume or response-time test result appears in any excerpt.
  • Unknown R6 Expected peak demand, tested concurrent users and SLA targets are not in the excerpts.
  • Unknown R6 Whether the state and federal partner SLAs planned for July 2013 were signed.
  • Unknown R7 Whether the GAO-listed CGI Federal contract is the same award CGI announced on 2011-12-01: the announcement gives no task-order identifier.
  • Unknown R7 Obligations after March 31, 2013 and any contract modifications.
  • Unknown R7 Which party is accountable for end-to-end integration and testing across contractors and partners.
  • Unknown ALL The excerpts are partial; full documents may contain evidence not reproduced here. This audit judges only what the excerpts show.

Sources

  • HC1 GAO-13-601: GAO-13-601 Status of CMS Efforts to Establish Federally Facilitated Health Insurance Exchanges. U.S. GAO, . Published June 19; findings mostly based on CMS records through early May.
  • HC2 CMS release: HHS launches Health Insurance Marketplace educational tools. CMS, . Press release with CMS launch plans.
  • HC3 CGI release: CGI selected to build U.S.-wide competitive health insurance exchange. CGI, . Contemporaneous contractor award announcement; not a final federal obligation.
  • HC4 House majority memo: PPACA Pulse Check Part 2 majority staff memorandum. House Energy and Commerce Subcommittee on Health, . Majority staff position; not GAO finding.

Reproduce

python3 audit.py python3 check.py python3 check.py --self-test

audit.py builds analysis.json and this page from the excerpt files. check.py re-parses the excerpts on its own and rejects mismatched values, a planned milestone shown as passed, a contractor estimate shown as spending, broken source links and any later outcome.

GAO page ranges are printed with an en dash in the excerpt file; locators here use an ASCII hyphen. Quotes are verbatim lines.