Dada Mail File Formats

- Dada Mail Specific File Formats
- List Subscribers

- PlainText
- Postgres
- MySQL
- See Also:

- List Settings

- DBM File
- Postgres
- MySQL

- Message Archives

- DBM File
- Postgres
- MySQL

- Scheduled Mailings
- Backups
- Usage Logs
- Error Logs
- Session Information

- See Also:

# Dada Mail Specific File Formats

The File Formats for much of Dada Mail's list settings, archives, etc are very basic. Having other programs read the information and use it should be relatively easy.

# List Subscribers

Currently, Dada Mail is shipped with support for three different types of backends for the subscription list. They are:

## PlainText

The default backend is, PlainText. Each list has one file, named, listshortname.list, where listshortname is the list's short name. This file is saved wherever the $FILES Config.pm variable is set to. This file is where the subscribers to your mailing list are saved.

The internal file format is simply one email address per line, with a newline (in Perl, ``\n'') separated each line. The end of the file has an extra newline.

For example:

     one@example.com
     two@example.com
     three@example.com

There are currently two other subscription lists that are associated with a Mailing List. They are the, black list and invitation list.

The blacklist is saved in a file with the same file format, except it has a, black_list file ending, example: listshortname.black_list.

The Invitation list has the same file format as the regular subscription list, except that it has a, invitelist file ending, example: <listshortname.invitelist>

## Postgres

Dada Mail also supports a Postgres based subscription list backend.

The specific table schema is described in the dada/extras/SQL/dada_subscribers.sql file in the Dada Mail distribution.

- email_id

  is an auto-generated ID number

- email

  Holds the actual email address

- list

  Holds the listshortname the email address is subscribed to

- list_type

  Holds either, list, black_list or <invitelist> (see the PlainText docs above)

- list_status

  Almost always is set to, 1. Setting it to something else won't have an affect you may intend.

## MySQL

Dada Mail also supports a MySQL based subscription list backend.

The specific table schema is described in the ada/extras/SQL/dada_subscribers.mysql file in the Dada Mail distribution.

The fields in the schema play the same role as what's described for Postgres, although their structure is slightly different, given the differences of the two SQL products.

## See Also:

- DADA::MailingList::Subscribers

  MailingList_Subscribers.pm.html

- DADA::MailingList::Subscribers::PlainText

  MailingList_Subscribers_PlainText.pm.html

- DADA::MailingList::Subscribers::baseSQL

  MailingList_Subscribers_baseSQL.pm.html

- DADA::MailingList::Subscribers::MySQL

  MailingList_Subscribers_MySQL.pm.html

- DADA::MailingList::Subscribers::PostgreSQL

  MailingList_Subscribers_PostgreSQL.pm.html

# List Settings

Currently, Dada Mail is shipped with support for three different types of backends for the List Settings. They are:

## DBM File

The default backend is, Db.

Dada Mail can support any DBM File format that's support by the CPAN Perl Module, AnyDBM_File. By default, the following types are support by default:

- DB_File
- GDBM_File
- NDBM_File
- ODBM_File

SDBM_File could also be used, but it's own file format limits the amount of information that can be stored per key value and Dada Mail's archived messages are known to go over this, fairly easily. The DBM file type is picked automatically, first trying DB_File, all the way to ODBM_File. You can set this precedence yourself in the Config.pm variable, @AnyDBM_File

More information:

Config.pm.html#_anydbm_file

Each list has one file, named, mj-listshortname, where listshortname is the list's short name. This file is saved wherever the $FILES Config.pm variable is set to.

The internal format of list settings is basically: Each key holds a the settings name, the value is the value of the setting itself.

The default key/values are saved in the Config.pm variable, %LIST_SETUP_DEFAULTS. Programically, only the keys that are listed in that hash can be saved in the list settings.

List Setting values are currently extremely free form. Most of them are simple strings, but some may be either encoded Perl complex data structures, different fields separated by a specific delimeter, or something quite different all together. For the most part, they are simple strings.

## Postgres

Dada Mail also supports a Postgres based list settings backend.

The specific table schema is described in the dada/extras/SQL/dada_settings.sql file in the Dada Mail distribution.

- list

  The list shortname associated with the list setting.

- setting

  The Name of the list setting

- value the value of the list setting

## MySQL

Dada Mail also supports a MySQL based list settings backend.

The specific table schema is described in the ada/extras/SQL/dada_settings.mysql file in the Dada Mail distribution.

The fields in the schema play the same role as what's described for Postgres, although their structure is slightly different, given the differences of the two SQL products.

See Also:

- DADA::MailingList::Settings

  MailingList_Settings.pm.html

- DADA::MailingList::Settings::Db

  MailingList_Settings_Db.pm.html

- DADA::MailingList::Settings::baseSQL

  MailingList_Settings_baseSQL.pm.html

- DADA::MailingList::Settings::MySQL

  MailingList_Settings_MySQL.pm.html

# Message Archives

Currently, Dada Mail is shipped with support for three different types of backends for the message archives. They are:

## DBM File

The default backend is, Db.

Dada Mail can support any DBM File format that's support by the CPAN Perl Module, AnyDBM_File. By default, the following types are support by default:

- DB_File
- GDBM_File
- NDBM_File
- ODBM_File

SDBM_File could also be used, but it's own file format limits the amount of information that can be stored per key value and Dada Mail's archived messages are known to go over this, fairly easily. The DBM file type is picked automatically, first trying DB_File, all the way to ODBM_File. You can set this precedence yourself in the Config.pm variable, @AnyDBM_File

More information:

Config.pm.html#_anydbm_file

Each list has one file, named, mj-listshortname-archive, where listshortname is the list's short name. This file is saved wherever the $ARCHIVES Config.pm variable is set to. If the $ARCHIVES variable isn't explicitly set, the value of the $FILES variable is used.

The internal format of archived messages is basically: Each key holds a simplified message-id for the email message information saved in the value.

The message id itself is based on the date the message is sent, and the time and date of when it was sent it taken from the key. More information on how this key is created can be found in the message_id() subroutine in DADA::App::Guts:

App_Guts.pm.html#message_id

The value saved is a string, with delimeters to separate the different types of information saved.

Because the age of this format and the various revisions, the format is quite dirty (for lack of a better word) and shouldn't be used as an example of the pinnacle of using the DBM File format for saving of complex information.

Anyways...

By default, the delimiter is the string, [::].

The information that's currently saved, in order of how it is seen in the string is:

- Subject

  The message Subject

- Message

  The actual email message.

  Note! That only the main message body is saved in the DBM backend. This backend is not at all MIME-aware, having been made long before Dada Mail become completely MIME aware itself. Look below for more limitations of this format.

- The Format of the email message

  Basically states if the email message is a PlainText message, or an HTML message. This field is fairly free form, being able to accept a Content-Type-like header (text/plain, text/html) or just something like, ``Text'', or, ``HTML''.

An example of the value of a key in the of an archived message DBM would look like this:

     My Subject![::]The actual text of the message![::]text/plain

Where, My Subject! is the Subject of the message, The actual text of the message! is the actual text of the message and, text/plain is the format of the message.

There are many limitations and known issues of this backend:

- [::] delimiter

  The format of the archived message will most likely break in some ways if either the Subject, Message Body or the Format contains the, [::] delimter. Currently, it doesn't look like there's any escaping of this character from any of the fields before saving.

- MIME messages simplified

  This particular backend type does not support saving of MIME messages for a few reasons. The headers of an email message are stripped and only a simplified form of the format of the body of the email message is saved (And this is only text or HTML). We haven't had good results in dumping the entire MIME message as the, ``Body'' field of this backend. For unknown reasons, this seems to cause the backend to be unstable. A workaround to that particular problem may be just to binary encode the message, before saving, but you'd also have to deal with backwards compatibility of older messages archived.

- Message size

  Some of the DBM backends only support a specific # of bytes per key value. This is most notable in the SDBM file type, which is now not currently supported by Dada Mail.

- key

  Keys are created using a simple format based on the date of the message sending. Date information isn't encoded in the value, so this is the only place where it's kept. If two or more messages have the exact same date, only one message will be saved, the other messages will be lost.

## Postgres

Dada Mail also supports a Postgres based archive backend.

The specific table schema is described in the dada/extras/SQL/dada_archives.sql file in the Dada Mail distribution.

- list

  The list shortname associated with the archived message.

- archive_id

  Similar to the keys of the DBM-based backend.

- subject

  The Subject of the message. This field is not necessarily filled in as the, raw_msg contains this information as well. Part of the table for backwards-compatibility.

- message

  A simplified version of the Message Body. Not necessarilly filled in, as the raw_msg field contains this information as well. Part of the table for backwards-compatibility

- format

  A simplified version of the Message Content-Type. Not necessarilly filled in, as the raw_msg field contains this information as well. Part of the table for backwards compatibility.

- raw_msg

  Containts the complete email message, including the email message headers.

A main advantage of using the SQL-based backend is that it suffers from almost none of the shortcomings of the DBM-based backend, save for the message-id collision problem.

## MySQL

Dada Mail also supports a MySQL based archive backend.

The specific table schema is described in the ada/extras/SQL/dada_archives.mysql file in the Dada Mail distribution.

The fields in the schema play the same role as what's described for Postgres, although their structure is slightly different, given the differences of the two SQL products.

See Also:

- DADA::MailingList::Archives

  MailingList_Archives.pm.html

- DADA::MailingList::Archives::Db

  MailingList_Archives_Db.pm.html

- DADA::MailingList::Archives::baseSQL

  MailingList_Archives_baseSQL.pm.html

- DADA::MailingList::Archives::MySQL

  MailingList_Archives_MySQL.pm.html

- DADA::MailingList::Archives::PostgreSQL

  MailingList_Archives_PostgreSQL.pm.html

# Scheduled Mailings

Currently, Dada Mail supports only one type of backend for its scheduled mailings. Scheduled Mailings itself are created using the Beatitude plugin.

The file type itself is also a DBM-based file format, with multiple fields for the value. The fields are managed a bit more sophisticatedly, using the CPAN, MLDBM. The value saved is a serialized Perl complex data structure, that looks a bit like this:

        my %d_form_vals = (
                message_name           => 'scheduled mailing',
                active                  => 0,
                mailing_date            => time,
                repeat_times            => 1,
                repeat_label            => 'days',
                repeat_mailing          => 0,
                only_send_to_list_owner => 0,
                archive_mailings        => 0,
                only_send_if_diff       => 0,
                self_destruct           => 0,

                PlainText_ver => {
                                   source                    => 'from_text',
                                   use_email_template        => 1,
                                   only_send_if_defined      => 0,
                                   grab_headers_from_message => 0,
                                   checksum                  => undef,

                                 },
                HTML_ver      => {
                                   source                    => 'from_text',
                                   use_email_template        => 1,
                                   only_send_if_defined      => 0,
                                   grab_headers_from_message => 0,
                                   checksum                  => undef,

                                 },
                attachments   => [],
            );

See Also:

- Beatitude Plugin

  scheduled_mailings.pl.html

- DADA::MailingList::Schedules

  http:///dadamailproject.com/support/documentation/MailingList_Schedules.pm.html

- DADA::MailingList::Schedules::MLDb

  MailingList_Schedules_MLDb.pm.html

# Backups

By default, automated backups are created for the List Settings, Archives and Schedules backends.

The List Settings are automatically backed up if the backend is the DBM file.

The List Archives are automatically backed up if the backend is the DBM file.

The List Schedules are automatically backed up if the backend is the DBM file.

Backups are not made for the subscription lists.

Backups are not made for any of the SQL backends.

The backups are file-based and OS platform neutral.

By default, these backups are stored in the directory set in the Config.pm variable, $BACKUPS. If this variable isn't explicitly set, the directory set in the $FILES variable is used.

The format seems complex at first, but is really quite simple:

Each list has its own directory, named after the list's short name. The contents of this directory will be three other directories, named, archives, schedules and settings - named appropriately for what information they save.

Contained in any one of those directories is a list of more directories, named after the UNIX `time()` they were created.

In any one of these directories is a list of files. Each file is named after the name of the key and its contents are the values from that key.

You can reload/reset your list archives/schedules/settings by visiting the following URL in your web browser:

http://example.com/cgi-bin/dada/mail.cgi?f=restore_lists

Where, http://example.com/cgi-bin/dada/mail.cgi

Is the URL of your Dada Mail installation.

For more information, see the dada/DADA/App/GenericDBFile/Backup.pm Perl module.

You can set how many copies of backups you'd like to save in the Config.pm variable, %BACKUP_HISTORY

Config.pm.html#_backup_history

# Usage Logs

The Dada Mail usage logs are saved wherever you've set the Config.pm variable, $PROGRAM_USAGE_LOG

The format is fairly free-form, but does follow some conventions, for example (Instead of newlines, tabs are used):

     [Tue Jan 16 22:07:54 2007]
     listshortname
     127.0.0.1
     Subscribed to dadatest.list
     email1@example.com

[Tue Jan 16 22:07:54 2007] Apache-style date of the action

listshortname the list's shortname of the list being worked with

127.0.0.1 the IP Address that the action was started by

Subscribed to dadatest.list the action itself

email1@example.com More information about the action, in this case, the email subscribed.

For more information:

Logging_Usage.pm.html

# Error Logs

The Dada Mail error logs are saved whereer you've set the Config.pm variable, $PROGRAM_ERROR_LOG.

The format is fairly free-form, but usualy has a Apache-style date stamp, followed by the error/warning reported by the program.

# Session Information

Dada Mail has two methods it can use to keep state for administrative control panel sessions.

By default, it uses the CPAN CGI::Session module and supports the following CGI::Session backends:

- PlainText
- Db (Berkeley DB File, in this case)
- MySQL
- PostgreSQL

The other method is currently known as, ``Classic'' and is basically the system that was in place before we moved to the CGI::Session module.

This method using no temporary session keeping file, but rather keeps all information in the web browser's cookie. Because this information includes a password (encrypted), it's not suggested that you use this method, but it's there for backwards compatibility.

## See Also:

- $SESSION_DB_TYPE Config.pm variable:

  Config.pm.html#_session_db_type

- CGI::Session

  http://search.cpan.org/~markstos/CGI-Session/lib/CGI/Session.pm

- dada/DADA/App/Session.pm