=head1 Changes from 2.4.0 to 2.4.1

=over 2

=item *

SECURITY: Handle the special filing of control messages into per-type
newsgroups more robust.  This closes a potentially exploitable buffer
overflow.  Thanks to Dan Riley for his excellent bug report.

=item *

Fixed article handling in innd so that articles without a Path header
(arising from peers sending malformatted articles or injecting
malformatted articles through rnews) would not cause innd to crash.  (This
was not exploitable.)

=item *

Fixed a serious bug in XPAT handling, thanks to Tommy van Leeuwen.

=item *

configure now looks for sendmail only in /usr/sbin and /usr/lib, not on
the user's path.  This should reduce the need for --with-sendmail if your
preferred sendmail is in a standard location.

=item *

The robustness of the tradindexed overview method has been further
increased, handling more edge cases arising from corrupted databases and
oddly-named newsgroups.

=item *

innd now never decreases the high water mark of a newsgroup when
renumbering, which should help ameliorate overview and active file
synchronization problems.

=item *

Do not close and reopen the history file on ctlinnd reload when the server
is paused or throttled.  This was breaking ctlinnd reload all during a
server pause.

=item *

Various minor portability and compilation issues fixed.  Substantial
numbers of compiler warnings have been cleaned up, thanks largely to work
by Ilya Kovalenko.

=item *

Multiple other more minor bugs have been fixed.

=item *

Documentation and man pages have been clarified and updated.

=back

=head1 Upgrading from 2.3 to 2.4

The inn.conf parser has changed between INN 2.3 and 2.4.  Due to that
change, options in F<inn.conf> that contain whitespace or a few other
special characters must be quoted with double quotes, and empty parameters
(parameters with no value) are not allowed.  INN 2.4 comes with a script,
B<innupgrade>, run automatically during C<make update>, that will attempt
to fix any problems that it finds with your F<inn.conf> file, saving the
original as F<inn.conf.OLD>.

This change is the beginning of standardization of parsing and syntax
across all of INN's configuration files.

The history subsystem now has a standard API that allows other backends to
be used.  Because of this, you now need to specify the history method in
F<inn.conf>.  Adding:

    hismethod: hisv6

will tell INN to use the same history backend as was used in previous
versions.  B<innupgrade> should take care of this for you.

ovdb is known to have some locking and timing issues related to how nnrpd
shuts down (or fails to shut down) the overview databases.  If you have
stability problems with ovdb, try setting readserver to true in
F<ovdb.conf>.  This will funnel all ovdb reads through a single process
with a cleaner interface to the underlying BerkeleyDB database.

If you use Perl authentication for nnrpd (if I<nnrpdperlauth> in
F<inn.conf> is true), there have been major changes.  See L<"Changes to
Perl Authentication Support for nnrpd"> in F<doc/hook-perl> for details.

Similarly, if you use Python authentication for nnrpd (if
I<nnrpdpythonauth> in F<inn.conf> is true), there have been major changes.
See L<"Changes to Python Authentication and Access Control Support for
nnrpd"> in F<doc/hook-python> for details.

If you use B<send-uucp>, it has been completely rewritten and now takes a
configuration file to specify its behavior.  See its man page for more
information.  If you use B<sendbatch>, it is no longer included in INN
since the new B<send-uucp> can handle all of the same functionality.

The wildmat API has been renamed (to uwildmat and friends; see uwildmat(3)
for the interfaces) to distinguish it from Rich $alz's original version,
since it now supports UTF-8.  This may require changes in other software
packages that link against INN's libraries.

If you are upgrading from a version prior to INN 2.3, see L<"Upgrading
from 2.2 to 2.3">.

=head1 Changes from 2.3 to 2.4

=over 2

=item *

IPv6 support has been added, disabled by default.  If you have IPv6
connectivity, build with --enable-ipv6 to try it.  There are no known
bugs, but please report any problems you find (or even successes, if you
use an unusual platform).  There are a few changes of interest; further
information is available in F<doc/IPv6-info>.

=item *

The tradindexed overview method has been completely rewritten and should
be considerably more robust in the face of system crashes.  A new utility,
B<tdx-util>, is provided to examine the contents of the overview database,
repair inconsistencies, and rebuild the overview for particular groups
from a tradspool news spool.  See tdx-util(8) for more details.

=item *

The Perl and Python authentication hooks for readers have been extensively
overhauled and integrated better with F<readers.conf>.  See the Changes
sections in F<doc/hook-perl> and F<doc/hook-python> for more details.

=item *

nnrpd now optionally supports article injection via IHAVE, see
readers.conf(5).  Any articles injected this way must have Date, From,
Message-ID, Newsgroups, Path, and Subject headers.  X-Trace and
X-Complaints-To headers will be added if the appropriate options are set
in F<readers.conf>, but other headers will not be modified/inserted (e.g.
NNTP-Posting-Host, NNTP-Posting-Date, Organization, Lines, Cc, Bcc, and To
headers).

=item *

nnrpd now handles arbitrarily long lines in POST and IHAVE; administrators
who want to limit the length of lines in locally posted articles will need
to add this to their local filters instead.

=item *

nnrpd no longer handles the poorly-specified RFC 977 optional fourth
argument to the NEWGROUPS command specifying the "distributions" that the
command was supposed to apply to.

Clients that use that argument will break.  There are not believed to be
any such clients, and it's easy enough to just filter the returned list of
newsgroups (which is generally fairly short) to achieve the same results.

=item *

nnrpd no longer accepts UTC as a synonym for GMT for NEWGROUPS or NEWNEWS.
This usage was never portable, and was rejected by the NNTP working group.
It is being removed now in the hope that it will be caught before anyone
starts to rely on it.

=item *

innfeed supports a new peer parameter, I<backlog-feed-first>, that if set
to true feeds any backlog to a peer before new articles, see
innfeed.conf(5).  When used in combination with I<max-connections: 1>,
this can be used to enforce in-order delivery of messages to a peer that
is doing Xref slaving, avoiding cases where a higher-numbered message is
received before a lower-numbered message in the same group.

=item *

Several other, more minor protocol issues have been fixed:  connections
rejected due to the connection rate limiting in innd receive 400 replies
instead of 504 or 505, and ARTICLE without an argument will always either
retrieve the current article or return a 423 error, never advance the
current article number to the next valid article.

See F<doc/compliance-nntp> for all of the known issues with INN's
compliance with the current NNTP draft.

=item *

All accesses to the history file for all parts of INN now go through a
generic API like the storage and overview subsystems do.  This will
eventually allow new history implementations to be dropped in without
affecting the rest of INN, and will significantly improve the
encapsulation of the history subsystem.  See the libinnhist(3) man page
for the details of the interface.

=item *

INN now uses a new parser for the F<inn.conf> file.  This means that
parameters containing whitespace or other special characters must now be
quoted; see inn.conf(5).  It fixes the long-standing bug that certain
values must be included in F<inn.conf> even if using the defaults for the
use of shell or Perl scripts, and it will serve as the basis for
standardizing and cleaning up the configuration file parsing in other
parts of INN.  B<innupgrade> is run during make update and should convert
an existing F<inn.conf> file for you.

=item *

B<send-uucp> has been replaced by a completely rewritten version from
Marco d'Itri, Edvard Tuinder, and Miquel van Smoorenburg, which uses a
configuration file that specifies batch sizes, compression methods, and
hours during which batches should be generated.  The old B<sendbatch>
script has been retired, since B<send-uucp> can now handle everything
that it did.

=item *

Two configure options have changed names:  --with-tmp-path is now
--with-tmp-dir, and --with-largefiles is now --enable-largefiles, to
improve consistency and better match the autoconf option guidelines.

=item *

Variables can now be used in the F<newsfeeds> file to make it easier to
specify many similar feeds or feed patterns.  See the newsfeeds(5) man
page for details.

=item *

Local connections to INN support a new special mode, MODE CANCEL, that
allows efficient batch cancellation of messages.  This is intended to be
the preferred interface for external spam and abuse filters like NoCeM.
See L<CANCEL FEEDS> in innd(8) for details.

=item *

Two new options, I<nfsreader> and I<nfswriter>, have been added to
F<inn.conf> to aid in building NFS based shared reader/writer platforms.
On the writer server configure nfswriter to true and on all of the readers
configure nfsreader to true; these options add calls to force data out to
the NFS server and force it to be read directly from the NFS server at the
appropriate moments.  Note that it has only been tested on Solaris 8,
using CNFS as the storage mechanism and tradindexed as the overview
method.

=item *

A new option, I<tradindexedmmap>, has been added to F<inn.conf>.  If set
to true (the default), then the tradindexed overview method will use
mmap() to access its overview data (in 2.3 you couldn't control this; it
always used mmap).

=item *

Thanks to code contributed by CMU, innfeed can now feed an IMAP server as
well as other NNTP servers.  See the man page for innfeed(8) for more
information.

=item *

An authenticator, B<auth_smb>, that checks a username and password against
a remote Samba server is now included.  See auth_smb(8) for details.

=item *

The wildmat functions in INN now support UTF-8, in a way that should allow
them to still work with most simple 8-bit character sets in widespread
use.  As part of this change, some additional wildmat interfaces are now
available and the names have changed (to uwildmat, where u is for
Unicode).  See uwildmat(3) for the details.

=item *

The interface between external authenticators and nnrpd is now properly
documented, in F<doc/external-auth>.  A library implementing this
interface in C is provided, which should make it easier to write
additional authenticators resolvers.  See libauth(3) for details, and any
of the existing programs in F<authprogs/> for examples.

=item *

INN now checks to ensure that the configured temporary directory is not
world-writeable.  Additionally, most (if not all) of the temporary file
creation in INN now uses functions that create temporary files properly
and safely.

=item *

All of the applicable bug fixes from the INN 2.3 STABLE series are also
included in INN 2.4.

=back

=head1 Upgrading from 2.2 to 2.3

There may be additional things to watch out for not listed here; if you
run across any, please let inn-bugs@isc.org know about them.

Simply doing a make update is not sufficient to upgrade; the history and
overview information will also have to be regenerated, since the formats
of both files have changed between 2.2 and 2.3.  Regardless of whether you
were using the storage API or traditional spool under 2.2, you'll need to
rebuild your overview and history files.  You will also need to add a
storage.conf file, if you weren't using the storage API under INN 2.2.  A
good default storage.conf file for 2.2 users would be:

    method tradspool {
        newsgroups: *
        class: 0
    }

Create this storage.conf file before rebuilding history or overview.

If you want to allow readers, or if you want to expire based on newsgroup
name, you need to tell INN to generate overview data and pick an overview
method by setting I<ovmethod> in F<inn.conf>.  See INSTALL and inn.conf(5)
for more details.

The code that generates the dbz index files has been split into a seperate
program, F<makedbz>.  F<makehistory> still generates the base history file
and the overview information, but some of its options have been changed.
To rebuild the history and overview files, use something like:

    makehistory -b -f history.n -O -T/usr/local/news/tmp -l 600000

(change the /usr/local/news/tmp path to some directory that has plenty of
temporary space, and leave off -O if you're running a transit-only server
and don't intend to expire based on group name, and therefore don't need
overview.)  Or if your overview is buffindexed, use:

    makehistory -b -f history.n -O -F

Both will generate a new history file as F<history.n> and rebuild overview
at the same time.  If you want to preseve a record of expired message IDs
in the history file, run:

    awk 'NF==2 { print; }' < history >> history.n

to append them to the new history file you created above.  Look over the
new history file and make sure it looks right, then generate the new index
files and move them into place:

    makedbz -s `wc -l <history.n` -f history.n
    mv history.n history
    mv history.n.dir history.dir
    mv history.n.hash history.hash
    mv history.n.index history.index

(Rather than .hash and .index files, you may have a .pag file if you're
using tagged hash.)

For reader machines, F<nnrp.access> has been replaced by F<readers.conf>.
There currently isn't a program to convert between the old format and the
new format (if you'd like to contribute one, it would be welcomed
gratefully).  The new file is unfortunately considerably more complex as
a result of its new capabilities; please carefully read the example
F<readers.conf> provided and the man page when setting up your initial
configuration.  The provided commented-out examples cover the most common
installation (IP-based authentication for all machines on the local
network).

INN makes extensive use of mmap(2) for the new overview mechanisms, so at
the present time NFS-mounting the spool and overview on multiple reader
machines from one central server probably isn't feasible in this version.
mmap tends to interact poorly with NFS (at the least, NFS clients won't
see updates to the mapped files in situations where they should).  (The
preferred way to fix this would, rather than backing out the use of mmap
or making it optional, to add support for Diablo-style header feeds and
pull-on-demand of articles from a master server.)

The flags for F<overchan> have changed, plus you probably don't want to
run overchan at all any more.  Letting innd write overview data itself
results in somewhat slower performance, but is more reliable and has a
better failure mode under high loads.  Writing overview data directly is
the default, so in a normal upgrade from 2.2 to 2.3 you'll want to comment
out or remove your overchan entry in F<newsfeeds> and set useoverchan to
false in F<inn.conf>.

F<crosspost> is no longer installed, and no longer works (even with
traditional spool).  If you have an entry for crosspost in F<newsfeeds>,
remove it.

If you're importing a traditional spool from a pre-storage API INN server,
it's strongly recommended that you use NNTP to feed the articles to your
new server rather than trying to build overview and history directly from
the old spool.  It's more reliable and ensures that everything gets put
into the right place.  The easiest way to do this is to generate, on your
old server, a list of all of your existing article files and then feed
that list to innxmit.  Further details can be found in the FAQ at
I<http://www.eyrie.org/~eagle/faqs/inn.html>.

If you are using a version of Cleanfeed that still has a line in it like:

    $lines = $hdr{'__BODY__'} =~ tr/\n/\n/;

you will need to change this line to:

    $lines = $hdr{'__LINES__'};

to work with INN 2.3 or later.  This is due to an internal optimization of
the interface to embedded filters that's new in INN 2.3.

=head1 Changes from 2.2 to 2.3

=over 2

=item *

New F<readers.conf> file (replaces F<nnrp.access>) which allows more
flexible specification of access restrictions.  Included in the sample
implementations is a RADIUS-based authenticator.

=item *

Unified overview has been replaced with an overview API, and there are now
three separate overview implementations to choose from.  One (tradindexed)
is very like traditional overview but uses an additional index file.  The
second (buffindexed) uses large buffers rather than separate files for
each group and can handle a higher incoming article rate while still being
fast for readers.  The third (ovdb) uses Berkeley DB to store overview
information (so you need to have Berkeley DB installed to use it).  The
I<ovmethod> key in F<inn.conf> chooses the overview method to use.

Note that ovdb has not been as widely tested as the other overview
mechanisms and should be considered experimental.

=item *

All article storage and retrieval is now done via the storage API.
Traditional spool is now available as a storage type under the storage
API.  (Note that the current traditional spool implementation causes
nightly expire to be extremely slow for a large number of articles, so
it's not recommended that you use the tradspool storage method for the
majority of a large spool.)

=item *

The timecaf storage method has been added, similar to timehash but storing
multiple articles in a single file.  See F<INSTALL> for details on it.

=item *

INN now supports embedded Python filters as well as Perl and TCL filters,
and supports Python authentication hooks.

=item *

There is preliminary support for news reading over SSL, using OpenSSL.

=item *

Users can no longer post articles containing Approved: headers to
moderated groups by default; they must be specifically given that
permission with the access: parameter in F<readers.conf>.  See the man
page for more details.

=item *

To simplify anti-abuse filtering, and to be more compliant with news
standards and proposed standards, INN now treats as control messages only
articles containing a Control header.  A Subject line beginning with
C<cmsg > is no longer sufficient for a message to be considered a control
message, and the Also-Control header is no longer supported.

=item *

inews is not installed setgid news and rnews is not installed setuid root
by default any more.  If you need the old permissions, you have to give a
flag to configure.  See F<INSTALL> for more details.

=item *

The INN build system no longer uses subst.  (This will be transparent to
most users; it's an improvement and modernization of how INN is
configured.)

=item *

The build and installation system has been substantially overhauled.
C<make update> now updates scripts as well as binaries and documentation,
there is better support for parallel builds (C<make -j>), there is less
make recursion, and far more of the system-dependent configuration is
handled directly by autoconf.  libtool build support (including shared
library support) should be better than previous releases.

=item *

All of the applicable bug fixes from the INN 2.2 STABLE series are also
included in INN 2.3.

=back

=head1 Changes from 2.1 to 2.2

=over 2

=item *

New storage.conf file (replaces storage.ctl).

=item *

New (optional) way of handling non-cancel control messages (controlchan)
that serializes them and prevents server overload from control message
storms.

=item *

Support for actsyncd to fetch active file with ftp; configured by default
to use <ftp://ftp.isc.org/pub/usenet/CONFIG/active.Z> if you run actsyncd.
Be sure to read the manual page for actsync to configure an actsync.ign
file for your site, and test simpleftp if you do not configure with wget
or ncftp.  Also see <ftp://ftp.isc.org/pub/usenet/CONFIG/README>.

=item *

Some options to configure are now moved to inn.conf (merge-to-groups and
pgp-verify).

=item *

inndf, a portable version of df(1), is supplied.

=item *

New cnfsstat program to show stats of cnfs buffers.

=item *

news2mail and mailpost programs for gatewaying news to mail and mail to
news are supplied.

=item *

pullnews program for doing a sucking feed is provided (not meant for large
feeds).

=item *

The innshellvars.csh.in script is obsolete (and lives in the obsolete
directory, for now).

=back
