Avoiding Vogon Poetry
Generally speaking, SubEtha tries to modify messages as little as possible
as they are passed through the system. Hopefully this means the receiver
will see the message in the same encoding and character set that it was
sent with. However, there are a few cases where this might not be the case:
- The SubEtha web interface always uses UTF-8. All contents of
Mail messages will be converted to this encoding before being
rendered as HTML.
- SubEtha allows list administrators to use plugin modules called
Mail Filters to arbitrarily manipulate the message stream. When
a mail filter adds text or replaces text in a header, it (almost)
always uses the Java platform default encoding. For example, a
SubjectFilter may receive a message with the subject header
encoded Shift-JIS, prepend "[ListName] ", and save the new header
encoded with UTF-8.
If you're running all your lists in your native language, you can
ignore any of these settings and everything will probably work
fine. However, we recommend the following, especially if you
plan to host lists in multiple languages:
-
Set the JVM platform default character encoding to UTF-8
This will cause any text that SubEtha creates or modifies
to be encoded in the most portable form possible.
HOW: In the jboss start script (look in ${jboss.dir}/bin/run.conf
or run.sh or run.bat), set JAVA_OPTS=-Dfile.encoding=UTF-8.
-
Make sure your database tables are UTF-8
While message content is stored as-is in the database as
BLOBs, the Subject is also kept in a separate column for
performance reasons. You'll want this to be encoded UTF-8.
Fortunately most modern databases use UTF-8 by default.
HOW: This is, unfortunately, very specific to the database
vendor. For MySQL, you want to include the following in your
my.cnf config file:
[mysqld]
default-character-set=utf8
Note that you must set this before you run
SubEtha for the first time. SubEtha creates the database
schema for you, and will happily create them with the wrong
charset. If you do this wrong, either manually change the
charset for each table (MySQL: alter table [tablename] character set=utf8)
or drop the database and start again (this will delete your data!).
-
Use substitution filters sparingly
The more you manipulate text, the more you risk replacing
the original encoding of a message. In particular, avoid
the Subject filter on lists that converse in multiple languages.
