Drat, pdftotext -enc Big5 drops several characters, e.g. of
$ qp-encode < a.txt
=A4T=A6~=A1A
the middle A6~ will disappear.  I bet it is the tilde. Nope, character
=B6q is also gone.
xpdf shows the character fine.
No I have no way to turn a.txt into a.pdf to test further.
All I know is that it happens in a file not on the net which is
$ pdfinfo zs.pdf
Title:          [some mess: pdfinfo doesn't know Big5!]
Creator:        Acrobat PDFMaker 6.0 for Word
Producer:       Acrobat Distiller 6.0 (Windows)
Tagged:         yes
Pages:          14
Encrypted:      yes (print:no copy:no change:no addNotes:no)
Page size:      595 x 842 pts (A4)
File size:      267329 bytes
Optimized:      yes
PDF version:    1.5

Too big to mail back and forth. I would send you the first page, but
$ pdftk zs.pdf burst
Error: Failed to open PDF file:
   zs.pdf
   OWNER PASSWORD REQUIRED, but not given (or incorrect)

Jeez how silly. Can't this be bypassed.

If you can upload the file by ftp to:

    ftp://ftp.glyphandcog.com/incoming/

then I'll take a look.

- Derek

One other thing on that -- can you point out exactly where you're seeing
the problem?  I can display CJK text files, but I don't actually read
Chinese, so I'll need to know exactly where to look.

- Derek

OK, I'll try to ftp it. zx.pdf.
Right in the very first line, xpdf shows all 11 chars, but pdftotext
-enc Big5 drops two chars!:

xpdf:      地理資訊更新與分享使用
pdftotext: 地資訊新與分享使用

Hey! I did pdftops; gv on it and:
There are 11 chars, but those two are now wrong chars!
All the Chinese chars in the document look like they have been thru the
washing machine, with 95% to %5 remaining per character! Like someone
stole an "i" but just left the dot.

Adding for reference:

Date: Mon, 11 Apr 2005 15:59:34 -0700 (PDT)

That file (AW.PDF) has vectorized all of the fonts, which is why it
looks fine.

My guess is that ghostscript is running into the same issue as FreeType.
Apple apparently has a patent which covers (some part of) the bytecode
interpreter used by TrueType font engines.  I'm not familiar with gs's
TrueType implementation, but FreeType ships with the interpreter
disabled by default, and that causes major problems with some CJK fonts
that use the interpreter to move sections (not sure if that's the right
word) of the characters around.  The interpreter was initially intended
(by the TrueType format architects) to be used for hinting, but was
later co-opted into use for other things (or at least that's the story
I've heard).  So FreeType without the interpreter works ok for *most*
fonts, but fails badly on a few.

As far as I know, Apple hasn't sued anyone over the patent(s).  But
FreeType is being (justifiably) paranoid, and shipping with the
interpreter off by default.  I haven't looked at ghostscript that
closely, but they may be doing something similar.

In any case, if this is in fact the relevant problem, I'm sure the
ghoscript folks are well aware of it.

- Derek

fixed 521453 3.02-9
fixed 171956 3.02-9
fixed 409510 3.02-9
fixed 349556 3.02-9
fixed 349096 3.02-9
fixed 202138 3.02-9
fixed 551863 3.02-9
fixed 548766 3.02-9
fixed 521453 3.02-9
fixed 456669 3.02-9
fixed 442328 3.02-9
fixed 421600 3.02-9
fixed 298757 3.02-9
fixed 354506 3.02-9
fixed 359937 3.02-9
fixed 362127 3.02-9
fixed 406256 3.02-9
fixed 406568 3.02-9
fixed 406810 3.02-9
fixed 408342 3.02-9
fixed 411597 3.02-9
fixed 413628 3.02-9
thanks


Since xpdf uses poppler and do not provide programs such as
/usr/bin/pdftotext, I am closing all related bugs on xpdf-utils.

Osamu

521453-done@bugs.debian.org, 171956-done@bugs.debian.org, 409510-done@bugs.debian.org, 349556-done@bugs.debian.org, 349096-done@bugs.debian.org, 202138-done@bugs.debian.org, 551863-done@bugs.debian.org, 548766-done@bugs.debian.org, 521453-done@bugs.debian.org, 456669-done@bugs.debian.org, 442328-done@bugs.debian.org, 421600-done@bugs.debian.org, 298757-done@bugs.debian.org, 354506-done@bugs.debian.org, 359937-done@bugs.debian.org, 362127-done@bugs.debian.org, 406256-done@bugs.debian.org, 406568-done@bugs.debian.org, 406810-done@bugs.debian.org, 408342-done@bugs.debian.org, 411597-done@bugs.debian.org, 413628-done@bugs.debian.org