Skip to content

feat: add hwp/hwpx support and fixtures for testing#2230

Open
kimtth wants to merge 1 commit into
microsoft:mainfrom
kimtth:main
Open

feat: add hwp/hwpx support and fixtures for testing#2230
kimtth wants to merge 1 commit into
microsoft:mainfrom
kimtth:main

Conversation

@kimtth

@kimtth kimtth commented Jul 22, 2026

Copy link
Copy Markdown

Why MarkItDown Should Support HWP and HWPX

What Are HWP and HWPX?

Hangul is a Korean word processor made by Hancom, first released in 1988. Its position in South Korea, particularly in government and education, reflects its early Korean-language support when foreign alternatives were more limited.

The estimate is necessarily speculative:

  1. Current flow: If 1 million public-sector employees create 1–2 HWP/HWPX files per working week and 600,000 businesses create 0.5–2, the annual flow is roughly 68–166 million files. Allowing 10–15% for other users gives about 75–191 million files per year. Schools are not counted separately because their staff overlap with the public-sector total.
  2. Historical total: Hangul was released in 1988, but adoption and office use were lower in earlier years. Treating the 38-year average as 35–55% of today's rate gives roughly 1–4 billion files created.
  3. Caveat: This counts new files, not unique surviving documents, and excludes revisions, copies, and converted files. The defensible claim is that the corpus is large, entrenched, and still growing.

Hangul has two document formats:

Format Extension Container Role
HWP .hwp Legacy binary format; uses an OLE Compound File container The long-established default format, still found throughout Korean archives and active workflows.
HWPX .hwpx ZIP package containing XML and related resources, based on OWPML The open-format successor, registered as Korean national standard KS X 6101, designed to be more machine-readable and portable.

HWP has been the standard for decades. HWPX was introduced with Hangul 2010 as the modernization path:

  • It is XML-based, easier to parse programmatically, and better suited to data exchange.
  • Hancom says fresh installs can default to HWPX, while HWP remains available. Hancom HWPX FAQ

Interesting Facts

Microsoft tried to acquire Hancom. In the late 1990s, Microsoft reportedly attempted to buy Hancom, the company behind Hangul. The deal did not go through, leaving Hancom independent and illustrating Hangul's strategic importance in the Korean market.

Hangul benefited from Korean text input. Korean characters are composed from phonetic components (jamo). Software historically used two approaches:

  • Johap (조합, jo-hap): characters are built up dynamically by combining jamo components as the user types. This is how Korean is naturally composed.
  • Wansung (완성, wan-seong): only pre-assembled, complete syllable blocks are stored, covering a fixed set of commonly used characters.

When Hangul rose to dominance:

  • Microsoft Word supported only Wansung encoding (KS X 1001), which could not represent every Korean syllable combination.
  • Hangul supported Johap and the full range of Korean characters.

This gave Hangul an important technical advantage before Microsoft Word caught up.

Why Both Formats Still Coexist

South Korea is actively migrating from HWP to HWPX, but the transition is gradual. A Korean document-ingestion pipeline will encounter both:

  • Legacy HWP is embedded throughout existing government archives, school communications, legal documents, and business files built up over decades.
  • New HWPX is increasingly required for new public-sector documents. Korea's central government On-Nara system made HWPX mandatory in 2022; local-government expansion began in May 2026.

Supporting only one format misses a large portion of real-world Korean documents.

Why MarkItDown Supports Both

MarkItDown extracts structured text for LLM and search pipelines. Korean primary-source documents often arrive as HWP or HWPX attachments, including:

  • Ministry press releases
  • Government contracts
  • Legal forms

The Korean government's AI strategy makes this more urgent:

  • A 2026-04-24 announcement from Korea's Presidential Committee on National AI Strategy identified binary HWP as harder for AI to analyze and HWPX as the preferred open format for the AI era.
  • HWP attachment restrictions in government mail channels are targeted for October 2026.
  • Supporting both formats covers the transition period and its emerging workflows. Source

Why rhwp-python Is the Backend

Discussion #1667 demonstrates the need for HWP-to-Markdown conversion in Korean government, legal, and academic workflows. Its Docpler-based plugin is a useful HWP-only option, but MarkItDown uses rhwp-python (rhwp-python>=0.8.1) because it is a better fit for built-in support:

  • Covers the transition: One backend handles both legacy HWP 5.0 and HWPX.
  • Fits MarkItDown: It is installed through the optional hwp extra and registered as a built-in converter, rather than requiring a separate plugin to be discovered and enabled.
  • Supports deployment: rhwp and rhwp-python use MIT licenses; the Docpler plugin uses Elastic License 2.0, which restricts hosted or managed-service use.
  • Targets extraction: It produces structured Markdown suitable for the RAG and LLM workflows described in the discussion.

rhwp-python is an independent community binding for the Rust-based rhwp project. It is not affiliated with Hancom; report parser-core defects to the upstream project.

Interoperability Context

HWP interoperability has existed for years:

  • Microsoft published a Korean-language HWP conversion tool for Word, useful as historical evidence of demand rather than a modern dependency. Microsoft Download Center ID 36772
  • ONLYOFFICE documented HWP/HWPX opening support in Desktop Editors 8.3, while noting that complex formatting may not convert perfectly. This reinforces the need for real-format regression fixtures. ONLYOFFICE: How to open HWP and HWPX files
  • LibreOffice includes import support for older Hangul WP 97 and HWP v5 files; recent builds can also open HWPX. These tools target visual import, not structure-preserving extraction, so complex layouts, tables, and embedded objects may not round-trip cleanly. LibreOffice: HWP/HWPX support

Public footprint signals, observed on 2026-07-22:

Channel Metric Source
Chrome Web Store 100,000 users; 322 ratings rhwp Chrome listing
Microsoft Edge Add-ons 5,471 users; 5 ratings rhwp Edge listing
Firefox Add-ons 270 users; 4 reviews rhwp Firefox listing
GitHub 3.6k stars, 634 forks, 43 contributors rhwp repository

These are snapshots, not a document-volume claim. The project does not publish opened or converted totals because its browser extension avoids analytics and tracking.

Reference Fixtures

These reference fixtures show the support decision against real Korean-government documents:

  • test_hwp.hwp — English translation of the standard residential lease agreement.
  • test_hwpx.hwpx — Ministry of Justice press release announcing translated standard residential-lease agreements.

Fixture provenance:

Fixture Official download URL Parsed Markdown PDF visual reference
test_hwpx.hwpx https://www.moj.go.kr/bbs/moj/182/491750/download.do expected_outputs/test_hwpx.md test_hwpx.pdf
test_hwp.hwp https://www.moj.go.kr/bbs/moj/182/491751/download.do expected_outputs/test_hwp.md test_hwp.pdf

These are content-bearing public documents, not synthetic samples. They cover common Korean administrative features:

  • Bilingual text and multi-page layouts
  • Tables, forms, and headings
  • Image references

Scope

  • HWP and HWPX are complex formats, so exact rendering, fonts, DRM/encrypted files, uncommon controls, and round-trip fidelity remain backend concerns.
  • Expected outputs are generated regression artifacts and should be refreshed deliberately when rhwp-python, its bundled rhwp core, or converter behavior changes.
  • This integration uses read-to-Markdown conversion only. It does not use the wrapper's HWPX writeback APIs.

References

@kimtth

kimtth commented Jul 22, 2026

Copy link
Copy Markdown
Author

@chienyuanchang
@afourney
Could you review this pull request? I've created a functional fix for these formats. This is not just a minor doc fix; I believe it can provide real value for MarkItDown.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant