The Health Internet vs. the NHIN — A Matter of Control, Cost, and Timing

David KibbeThere is growing tension within the Obama administration’s health team over who will control health data exchange: everyone (including consumers and their doctors), or just large provider organizations. The public debate will be framed in terms of privacy, security, and the adequacy of current exchange standards. But what really matters is who gets to make decisions about where health data resides, how it can be accessed, how much exchange will cost, and how long it will take for exchange to become routine.

Now is a good time to re-visit the plans for a National Health Information Network (NHIN), since we can finally observe and compare different health data sharing and exchange models in the marketplace. NHINs represent an older model that tries to use regional health information organizations (RHIOs) to establish secure networks, privately owned and operated by large provider organizations, mostly hospitals and health systems. The idea was that, over time, each private regional network would develop a gateway to other networks, creating a “network of networks” that would allow Stanford to talk to Partners Health, or Kaiser to Mayo. This communications model was enterprise/provider-centric. Patients/consumers were relegated to depending upon each RHIO’s policies for access to their health information. It was also a massively expensive and time consuming – think decades – way to build a health data network.

Suppose a RHIO is in your area. Your health data from hospitals, outpatient clinics, and other settings associated with Health System A, are collected and combined with health data stored in similar settings in Health System B. Possibly Health Systems C, D, and E have also collaborated with A and B in this RHIO. Most RHIOs have cost or will cost many millions of dollars to build and operate. They were greatly encouraged by the Office of the National Coordinator under the Bush Administration, and have received additional support and funding under the ARRA/HITECH provisions that establish Health Information
Exchanges (HIEs). They generally create large database management systems housed in large data centers. They typically run on proprietary software, creating closed networks that may or may not permit access onto and off the Internet.

As an individual, you probably don’t have direct access to the RHIO data; only doctors and nurses are authorized to access your information. In most RHIOs, if you request access to your health information you must make the request the same way you would to your physician’s medical practice, and often you will receive the results on paper. Transfer of these medical records to another institution or to a new provider outside the RHIO is not possible in most cases, although some RHIOs and HIEs now permit patient accounts and viewing of selected data.

By contrast, the Health Internet is a more current model, centered on the patient/consumer. As the name implies, the Health Internet leverages the Web’s physical network and its open protocols and standards for health data exchange controlled by patients (and/or patient agents, like doctors, through authorized web services). The idea is to develop mechanisms that allow health information to pass easily across institutional and business boundaries, to anywhere it’s needed. The Health Internet builds on the same Internet infrastructure and conventions that under-gird the transactions of major industry sectors like banking, e-commerce, retail sales, home mortgage business, and media and entertainment. Because this infrastructure is largely already in place, although little-used by health care entities now, the Health Internet could grow and scale rapidly at very little cost.

You can already see how the Health Internet is developing. You go to a CVS MinuteClinic, or to a handful of doctors, hospitals, labs, or pharmacies that offer you a personal health record account that lets you transfer your data in machine-readable format at will. You also create a Google Health account (or Microsoft HealthVault, Keas, or any number of personal health record platform websites) which allows you to upload your machine-readable, structured health data to them.

Next, you give your Google Health account permission to transfer your summary health data: to a doctor in anticipation of a visit; to a family member who is helping look after you; to a service that offers decision support based on your information to help you solve some of your health/wellness problems; or to a service that will organize your health data into folders categorized by date, or provider, or episode of illness. The important thing here is that you, the individual, are deciding when, why, and where your health information is going.

The Health Internet example we’ve described above is performing the foundational transactions required of a national health information exchange network, and is doing so today. There are many examples, and they are growing organically, without government support, without new and complex standards, and at very low cost.

Even so, the Health Internet’s growth is constrained mainly by the limited data available to patients and consumers from their doctors and hospitals, who continue to resist the idea that individuals ought to control their own data. They are also inhibited by patients’ reluctance to challenge their doctors and hospitals on this point.

These and other barriers also make the Health Internet an imperfect solution to the goals of secure and efficient interoperable health data transfer. For example, current coding and classification systems remain a complex stumbling block to any model of health data exchange. Various coding systems are in use. Some are proprietary and require pay-for-use, and others need to be extended and gain industry consensus to be truly useful.

But it is no coincidence that the British government is investigating using both Google Health and Microsoft HealthVault for personal health data exchange, moving away from its own National Health Service program, after the latter spent billions on a national information network that doesn’t appear to work. The NHIN “network of networks” model in this country is beginning to flounder, too, and may never achieve its future potential as a national system. The reasons are partly political, economic, and technological. An NHIN system’s triple burdens – smoothing over competitive markets, enormous cost, and proprietary complexity – created so that large systems like the VA and the DOD, Kaiser and Geisinger, can exchange data without having to reach the Internet, will likely sink this ship even before the British program runs aground.

The Health Internet, on the other hand, has the obvious advantage of not “re-inventing the wheel.”  As former Intel CEO Craig Barrett famously said, “We already have a network for health data, it’s called the Internet.”  Proponents of the Health Internet argue that, while health data and privacy and security are very important, the data themselves are inherently no different from financial data or the kinds of personal information routinely — and very securely — transported over the Internet using fair market encryption and other security technologies to protect it from intrusion, capture, or breach.  So why go backwards to create the equivalent of Prodigy or AOL in every state?  It could take forever.

We want to give credit to David Blumenthal, the Obama health team members and the folks at HHS who are taking a hard look at how best to create a secure and efficient method for health data transfer in this country.

David C. Kibbe MD MBA and Brian Klepper PhD write together on health care market dynamics, technology, policy and innovation.

Livongo’s Post Ad Banner 728*90

Categories: Uncategorized

Tagged as: , , , ,

22 replies »

  1. Look at http://france.angloinfo.com/countries/france/healthterms.asp for information on the Carte Vitale.
    Many people seem to believe patient’s medical records are being stored directly on the card, but it’s not actually feasible to do that, because the cards don’t have very much storage. (And what would happen to your lifetime’s worth of medical hsitory if you lose the card?)
    The Carte Vitale is a common “smart card” which is being used as a secure credential; what it carries is the information that a provider needs in order to access the bearer’s medical records and bill for whatever care they provide. The actual medical records are stored somewhere else (large, redundant, well-protected database servers belonging to the French National Health Service would be my guess).

  2. Plans for the National Health Information Network (Nhin), Organizations Regional Health Information (RHIOs) can be compared to large patent discoveries: few know, many needs and interests are conflicting. Health data available in small or large networks, should be a matter of common sense and not an act of obligation. Everyone needs, everyone can use at some point in their lives the data belonging to other groups. Important whether it is available!
    Dr. Silvio Sandro Cornelio – BRAZIL

  3. Hello I was looking at your blog and I find very interesting and entertaining, especially if the information is detailed and accurate, I hope you continue posting more items for report and comment.

  4. I also would to clarify something by the first post for this article. I think the first post that mention “PHIN” may be confusing in that the PHIN is a “public network” and refers to a technology method or using the Internet as the health network instead of a privately run network.
    There is actually a define architecture called “PHIN” or the national Public Health Information Network as in population health and the 10 core services that public health –nation, state, local, tribal government entities provide to the general population. i.e., Early Event Detection, Health Alerting, Countermeasures/Response (vaccinations), surveillance for potential threats (bioterrorism, pandemics, etc.) etc.…
    The PHIN was developed by the Center for Disease Control (CDC) and other national partners at the state, local, professional levels back around 2005. I think the genesis was actually the Health Alert Network (HAN) which CDC funded states to develop even earlier in the late 1990’s. After 9-11, Congress funded CDC to design and develop a national public health network and infrastructure to support five areas of public or population health disaster management; Early Event Detection (EED), Counter Measures and Response Administration (CRA), Connecting Lab Systems (CLS), Outbreak Management Systems (OMS) and Partner Communication and Alerting (PCA)-PCA covers HAN. PHIN 1.0 was very specific to the above functional domains and activities for supporting those areas in public health at the national, state, local levels.
    After the NHIN was initiated by the Bush Administration and charged HHS to deliver, CDC (an agency within HHS) needed to reorganize and align PHIN 1.0 with the NHIN strategy. Thus PHIN 2.0 was born in 2007 and CDC began working with States to comply with the architecture strategy. PHIN 2.0 is a technical architecture that covers all public or population health activities and is much less focused on specific functional requirements but instead focuses on data/vocabulary, messaging and privacy and security standards appropriate to the general healthcare domain defined by NHIN and the Federal Health Architecture (FHA) (HITSP, HL7, NIST, etc.).
    I hope this helps clarify what PHIN is and helps to correct/prevent the sometimes confusion in semantics that PHIN refers to a public network and the Internet with PHIN Architecture used by CDC and state, local, tribal public / population health entities.

  5. I think this discussion of the NHIN RHIO model vs. a “Health Network” ( Centralized vs. Federated model) points to a very important principle in IT and Interoperable, that is, “policy should precede technology”.

  6. Record keeping is purposefully placed out of reach of consumers and as a measure to restrict consumers from gaining information. Copies of records come at 35 cents a Page. Who’s records are they?
    While Hospitals and Doctors are going to electronic Data and HIPPA has Set Barriers that some call protections. The Consumer hasn’t been given access to their own Data.Private Electronic accounts should be afforded to individuals to check and verify the Information is Correct. This would in most cases prevent medical Error and by providing the option to refute inaccuracies would provide greater control to the Patient.
    After all the information is shared with Law enforcement, Home Land Security, Insurance and third party organizations. I know for a fact transcription is done in India. The industry outsources nearly everything but it appears that they fear accountability and transparency. Deadly and Deceptive practices lay in the pages of Medical Records.

  7. Dr. Beller,
    I am not at all saying that it’s impossible to create secure and complete access to medical records.
    [I don’t really know what is meant by “control”. My experience in this industry has been that every time someone wants to give you “control” or “empower” you in some way, the net results are that you pay more and/or work more, usually for someone else’s benefit.]
    I am only suggesting that whatever other considerations may come into play, simplicity and ease of access have to be at the top of the list. That goes for both patients and doctors.

  8. Margalit,
    Are you saying that is isn’t possible for simple data privacy mechanisms to be created that give consumers control over their personal health information?
    It seems to me that with a little creativity and adequate field testing, PHRs can accomplish all that’s required. Although I can’t imagine how to deal with the complexity you noted using a purely centralized, monolithic cyber-infrastructure, I can envision how to do it via simple P2P pub/sub node networks.
    Let’s take the medical home model, for example. Every PCP (GP) establishes a community of referral, i.e., specialists to whom s/he refers patients as needed. The PCP and specialists would establish connections between their decentralized pub/sub nodes, which would enable them to exchange patient data with a few button clicks. The node-based software they use would automatically populate lists of these network connections; by using the e-mail based system I’ve been presenting, the lists would need little more than each specialist’s name, e-mail address, area clinical licensure, and other possible metadata.
    Prior to making a referral, the PCP would discuss with the patient why the referral is being made and explain why a particular specialist is being selected, just like things are currently done. Although no authorization by the patient is needed at this point, the patient may request a different specialist for whatever reason. The PCP would then click a button and the referral e-mail is sent.
    Once the PCP receives the specialist’s referral acceptance e-mail, the data for a CCR or CCD (or some similar data set) would be sent in an encrypted data file via e-mail to the specialists. But prior to sending it, the PCP’s node software would determine which data appropriate for that specialist must be excluded from the data file based on the patient’s privacy wishes. These data sharing authorizations would have previously come from the patient’s PHR by having the patient’s node send that information to the PCP’s node at an earlier date. The patient would establish the authorizations by, for example, (a) viewing lists showing the types of data that are appropriate for particular types of specialists (and why they are needed) and (b) enabling the patient to modify the list at any time (with appropriate warnings when data elements are deselected). The lists could be organized hierarchically to ease the viewing and selection process. It would even be possible (although I don’t know if necessary) to have the data set descriptions e-mailed to the patient for approval prior to routing the data file to the specialist.

  9. This is a demo of the first e-health service (see http://www.senscare.com ) that really measures your physiological signals. It uses your webcam and transform it into a health sensor. Based on optical propagation in biological tissues, it captures skin reflected light to extract hemoglobin variations and thus cardiovascular activity.

  10. Dear David,
    I’m all for Internet users in rural Montana, or Vermont where I live, having access “to information and knowledge on the human body and mind, health and wellness, and how to treat conditions and illnesses.” Indeed, the Internet is a wonderful source of information and knowledge. But it’s not a panacea.
    In my opinion, using the Internet to share individuals’ medical records is the wrong solution to what we all agree is an urgent problem. The risks are too great. Fortunately, there is no need to take them. We can accomplish the same ends more easily and cheaply with simpler, safer means!
    Accordingly, I’m not prepared to make the leap you and others insist upon making, namely, that because the Internet is there we should embrace it as the preferred medium to give individuals access to their medical records, or care providers access to their patients’ medical records.
    I’m all for a patient-centered medical record system. But we can give patients both access to, and control of, their records without exposing the them to the security and privacy risks inherent in Internet-accessible aggregated personal health records.
    If you haven’t seen it, I urge you to read Thursday’s (11/19/09) NY Times article about efforts to “Fend off Identity Fraud.” It starts with a statement from a research study that “9.9 million Americans were victims of identity theft in 2008, up from 8.1 million in 2007.” It ends with a statement from the CEO of a company trying to prevent such fraud: “Thieves can’t steal what isn’t there.”
    A key premise underlying the MedKaz™ personal health record system we are building is just that. If we don’t store masses of patient medical records on Internet-accessible servers, they can’t be stolen — so we don’t! We give them to the individual to own and control.
    Regards, Merle

  11. I think everybody can agree that patients have a right to see all their medical data and a right to decide who can see what portions of it and be notified of all disclosures of their medical records. I also think that HIPAA already mandates this. The implementation is of course spotty at best.
    My pain point with these new proposals is very simple. It is way too complicated. We are trying to replace a paper system, which today accomplishes all this data sharing by fax or copier and from a patient and doctor perspective it’s a very simple process.
    I know that many patients have difficulty obtaining copies of their medical records, but that has very little to do with the record being paper or electronic.
    Internet banking was adopted because it simplified the tasks. Instead of using calculators and writing checks and licking envelopes and stamps, you just click on a couple of check boxes. And it’s optional. I don’t have to do online banking if I don’t want to. The system is computerized with or without my participation.
    Unless, we make Internet healthcare equally simple for both doctors and patients, it will not gain adoption. As simple as that. There has to be a hard. measurable advantage to going electronic, or we will never get enough adoption to make it worthwhile.
    One of the main reasons doctors are not jumping on the EHR bandwagon is the inherent complexity and the lack of proven hard ROI to the doctor. I submit that the same will happen with consumers and PHRs.
    Some folks will be (are) intrigued, others prodded by the insurer and sign on initially. Most will not and many will drop out.
    A small minority will use it and love it. These are the same people that are running into access problems today. The PHR is offering a solution for them, but how many folks like that are there? Enough to satisfy Google’s business model? I seriously doubt that.
    The PHRs that are discussed here and elsewhere require patients to take control of the data. That means setting up the PHR, coming up with provider lists and entering them in the software with proper authorizations for various levels of access. Keeping these authorization lists current. Managing one’s credentials and also family members credentials. Making sure that all is up to date. Changing authorizations to various providers and care givers based on changes in health status and on and on….
    This sounds like a lot more work than most people need to do, or are doing, now.
    I’m certain that having PHRs is better for patients, just like I am certain that having EHRs is better for doctors. I am equally certain that products that complicate a working system by creating more work for people will not be successful, no matter how cool they are, or how hard they are pushed by well meaning interested parties.

  12. Dear Colleagues: Thanks for your thoughtful comments. This is a topic that gets people excited, and wanting to express their ideas, because I think we sense that health care can be less a commercial venture and more a cooperative endeavor.
    The Internet user in rural Montana has greater access to information and knowledge on the human body and mind, health and wellness, and how to treat conditions and illnesses, than anybody living in the 1960’s ever did. And more than most doctors today.
    What Internet users don’t have, generally, is acces to their OWN personal health data. This is because the health care institutions have changed very little, at least in terms of information management, since the 1960’s.
    It is really very difficult for anyone still to argue that keeping patients/consumers in the dark about their health data and information is good for them or good for society. We are seeing how the status quo is linked to speical interests and our inability to improve care or lower costs.
    We all sense that the change is coming, and that it will be a better world when it does.
    Kind regards, DCK

  13. With all due respect, ignoring consumer concerns about the security and privacy of Internet-accessible medical records doesn’t make the concerns go away or the records more secure. Neither does claiming they are secure when they are not.
    If your employer learns you have serious health or psychiatric issues, the probability is you’ll lose your job and your insurance — and have a helluva time getting another job or insurance. These are the kinds of real fears people have and, in my opinion, none of us concerned with improving healthcare IT should ignore or dismiss them as irrelevant. Our challenge is to recognize them and come up with a better solution.
    Neither should we take comfort from the oft-repeated but grossly inappropriate comparison of medical records to financial records. That’s not even comparing apples versus oranges. It’s sense versus nonsense!
    What financial record system accessible over the Internet (or otherwise) brings up the records of all your accounts at the one or more banks you deal with? Where can the banker considering your loan or mortgage application access all your bank accounts, brokerage accounts, credit card accounts, 401Ks, pension plans, asset appraisals, safe deposit boxes, etc. to analyze your financial resources and earnings stream? They can’t even if you authorize them to do so because no such system exists — for good reason. The public wouldn’t tolerate it!
    And what system can the Treasury Department, Federal Reserve System, IRS and other Federal and state agencies go to access these same records of yours and aggregate them with similar records from the rest of the population — so they can forecast government revenues, identify weaknesses developing in the economy, determine the appropriate interest rates needed to curb inflation or stimulate the economy? They can’t. Again, such a system doesn’t exist; the public wouldn’t tolerate it.
    Yet, isn’t that what you want to do in healthcare? Under the guise of improving the health of individuals and the public at large, you want to create a system by which care providers can access and aggregate individual patient records! Care to predict how the public will respond?
    I submit there is a simpler, cheaper, better way. Instead of storing a patient’s medical records on Internet-accessible servers or making it possible to aggregate them via the Internet, give patients physical control of their aggregated medical records. They’ll decide how they want them used. They’ll give them to their care providers when the providers need them — and they’ll get the improvements in care and reductions in costs they and we want but without the risks and costs associated with Internet-accessible records.

  14. Dave,
    Thanks for starting this great cascade of comments. I fundamentally agree that a person-oriented approach to health information exchange is likely to be the best way to get information flowing and manage the flow in a way that benefits health. I’ve led a group for the last 18 months who are planning such a system. The URL above is a link to the wiki for the project. I would like contribute a few points :
    – An approach that lets patients control the exchange process allows each individual to manage the process to his/her liking – and to change it as needs change. This avoids the traditional problem of having exchange throttled by third-party disclosure laws and takes advantage of laws/regulations that compel the production (most recently in HITECH – the transmission) of ePHI at patient request. This allows privacy conservatives (akin to the example of the person who hid their money in cash under a building) to have their way without making the vast majority (privacy rationals) have our information flow constrained more than we wish by a lowest common denominator type of regulatory regime.
    – A Health Internet approach offers an opportunity to embellish the doctor-patient relationship. For example, a typical person’s sharing rule (as applied by a software agent) might be “Here is my doctor. He is technically allowed to send PHI to anyone he wishes and these parties may respond to him.” If, at a later time, the patient wishes to change this rule, he/she can. But, while in effect this makes exchange easy for providers and keeps the patient in the loop. This also offers protections against medical identify theft. Requests from non-permitted entities would be denied and even for those who are permitted, the watchful eye of even a small percentage of patients on the exchange activity provides more audit power than e would otherwise be affordable.
    -For my taste, the question of whether most patients would use a health internet has been answered by the high use of the PHR system build by Kaiser-Permanente over the last 3-4 years. They now have about 50% penetration (routine users) in every demographic of their patient population – well except for teenagers. Apparently, they are immortal 😉
    – Lastly, recall that the highest objective of our work is to improve health. Improving health care (with HIE or anything else) is intended primarily to improve health. Engaging patients in care has been shown in a variety of settings to improve health; AHRQ posts a number of such studies. While using HIE to improve acute care is a worthy goal, so few of the determinants of health are routinely influenced by acute care, that we will leave a great deal of opportunity on the table if we only focus health internet efforts on improving acute care. The health internet opens the door to supports for improving health beyond acute care while integrated with acute care and should be valued for this reason. For my taste, RWJ’s Project HealthDesign as a great place to see this point in the flesh.

  15. Leonard makes good sense. Controlling information access, and authorizations constraining that access, are primary concerns in a collaborative environment such as healthcare where privacy and security are paramount.
    With good objective guidance, I contend, patients CAN actively manage their health information and determine the rules by which privacy is maintained. Simple defaults, which patients can override, are part of it. Warnings when patients prevent certain people from viewing/sharing information that’s critical to their care is another. Enabling them to have a granular level of privacy control is another. Using the cloud with caution and taking advantage of local encrypted data stores is another. Addressing all these issues technologically is feasible and need not be overly complex. But it will likely take disruptive innovation that minimizes cost and complexity. I believe this is less of a concern in countries where privacy issues are less important (e.g., China?).

  16. Dr. Kibbe,
    I think you are right that ” individuals (patients/consumers/citizens/people) ought to be able to control their own health data.” So, the health internet really comes down to who can access what information where and when, and who controls the authorizations.
    Yes, they ought to be able to control, but must they have to decide who has access and who can control authorizations?
    Is it realistic for all patients to actively manage their health information or determine the rules, particularly as we move to into true online collaboration in medicine and patient data is needed by countless different roles?
    I read today via Vince Kuratis that 67 clinicians will touch a record in an average 3-day stay. To keep it simple, we’ll need some reasonable defaults.
    Early indications of folks managing their data have not been all that impressive. PHRs have not seemed to have been met with wild success, mostly because they aren’t used by physicians and an overall lack of interest by people who aren’t sick or don’t have others to care for. Perhaps this will change. But I think maybe the basic crux is that people see a lot of risk in sharing this info online, but few benefits. As all things do, this will all ultimately come down to economics, trust and reward ahead of technology. As you point out, people (physicians, hospitals and patients) will share when they have the incentives to do so.
    Anyway, the point is that yes, it’a as simple as authority, but authority and control, but these things can get very, very complex, as has already been indicated by discussions in the Clinical Groupware Collaborative Group.
    How do we take a step beyond “let the patient decide?” A worthy goal, and it sound good, but what will it look like?
    (BTW- This could all be a paper tiger, not that it should be. We could give authorizations TOS-style, which nobody will ever read and everyone will agree to. If it comes to providing them care, they’ll likely give their precious medical info to just about anyone. No denial for pre-existing conditions will solve a big piece of that problem, then we’ll just have snoopy employers and neighbors to worry about.)
    What will an end solution look like? Great speculative post (http://www.shirky.com/weblog/2009/11/a-speculative-post-on-the-idea-of-algorithmic-authority/) today on what we may be driving toward by Clay Shirkey. Points to the larger issues at hand when we move toward sharing information and trusting information across a unified, coordinated data store. Ultimately, we trust what we know, and our trust is always imperfect. Our brains have pretty complex trust algorithms, and our systems will probably need them, too, to maximize value.
    Let’s not forget that medicine is by it’s very nature a collaborative endeavor, whereas our banking system is not. Health care requires literally hundreds of different people often to review a person’s info. Also, there are very clear reasons to share information in the banking system, and very clear benefits. While my bank info may be between me and my bank, and other systems may be entrusted to carry out specific functions with specific data sets, really only me and my bank have full access.
    What is it about this blog that gets people to write so profusely? I tried not to, but here we are. Thanks for hearing me out if you go this far. If you want more tangentially related to this topic, see my post from yesterday on the social sphere and the impact on health care:http://www.leonardkish.com/will-the-social-web-save-health-care-and-the

  17. China is also asking these questions and I have been soliciting interest among backers of an Internet approach to come to 3 meetings next year where this will be discussed. The top university in China is doing a conference which will include HIT for the first time. Also, the upcoming first HIMSS China meeting in May is kind of a watershed to see which way the winds are blowing for a massive government influx.
    Google and Microsoft are huge in China, and HealthVault just got a huge national contract. But Perot Systems also just got a contract to do an entire region RHIO. This in a system where there is no money for patient care! I am thinking the Internet way is the only way.
    If anybody here is interested in getting involved in the China question let me know.
    Bill Boyles, Publisher, Interpro Publications, Washington
    Also Director, Global Business Forum on Health helping produce conferences in Chna and Europe

  18. chronic pain narcotic opioids are effective but very dangerous, should be taken with moderation and prescribed by a doctor, medicines like hydrocodone, lortab, vicodin, norco, percocet, oxycontin, are even more commercial and very helpful to people with diseases such as fibromyalgia, chronic pain, Parkinson’s, arthritis, should be restricted and controlled as in findrxonline said that the FDA does not permit free marketing for them.

Leave a Reply

Your email address will not be published.