Annotation of embedaddon/rsync/TODO, revision 1.1
1.1 ! misho 1: -*- indented-text -*-
! 2:
! 3: FEATURES ------------------------------------------------------------
! 4: Use chroot only if supported
! 5: Allow supplementary groups in rsyncd.conf 2002/04/09
! 6: Handling IPv6 on old machines
! 7: Other IPv6 stuff
! 8: Add ACL support 2001/12/02
! 9: proxy authentication 2002/01/23
! 10: SOCKS 2002/01/23
! 11: FAT support
! 12: --diff david.e.sewell 2002/03/15
! 13: Add daemon --no-fork option
! 14: Create more granular verbosity 2003/05/15
! 15:
! 16: DOCUMENTATION --------------------------------------------------------
! 17: Keep list of open issues and todos on the web site
! 18: Perhaps redo manual as SGML
! 19:
! 20: LOGGING --------------------------------------------------------------
! 21: Memory accounting
! 22: Improve error messages
! 23: Better statistics Rasmus 2002/03/08
! 24: Perhaps flush stdout like syslog
! 25: Log child death on signal
! 26: verbose output David Stein 2001/12/20
! 27: internationalization
! 28:
! 29: DEVELOPMENT --------------------------------------------------------
! 30: Handling duplicate names
! 31: Use generic zlib 2002/02/25
! 32: TDB 2002/03/12
! 33: Splint 2002/03/12
! 34:
! 35: PERFORMANCE ----------------------------------------------------------
! 36: Traverse just one directory at a time
! 37: Allow skipping MD4 file_sum 2002/04/08
! 38: Accelerate MD4
! 39:
! 40: TESTING --------------------------------------------------------------
! 41: Torture test
! 42: Cross-test versions 2001/08/22
! 43: Test on kernel source
! 44: Test large files
! 45: Create mutator program for testing
! 46: Create configure option to enable dangerous tests
! 47: Create pipe program for testing
! 48: Create test makefile target for some tests
! 49:
! 50: RELATED PROJECTS -----------------------------------------------------
! 51: rsyncsh
! 52: http://rsync.samba.org/rsync-and-debian/
! 53: rsyncable gzip patch
! 54: rsyncsplit as alternative to real integration with gzip?
! 55: reverse rsync over HTTP Range
! 56:
! 57:
! 58:
! 59: FEATURES ------------------------------------------------------------
! 60:
! 61:
! 62: Use chroot only if supported
! 63:
! 64: If the platform doesn't support it, then don't even try.
! 65:
! 66: If running as non-root, then don't fail, just give a warning.
! 67: (There was a thread about this a while ago?)
! 68:
! 69: http://lists.samba.org/pipermail/rsync/2001-August/thread.html
! 70: http://lists.samba.org/pipermail/rsync/2001-September/thread.html
! 71:
! 72: -- --
! 73:
! 74:
! 75: Allow supplementary groups in rsyncd.conf 2002/04/09
! 76:
! 77: Perhaps allow supplementary groups to be specified in rsyncd.conf;
! 78: then make the first one the primary gid and all the rest be
! 79: supplementary gids.
! 80:
! 81: -- --
! 82:
! 83:
! 84: Handling IPv6 on old machines
! 85:
! 86: The KAME IPv6 patch is nice in theory but has proved a bit of a
! 87: nightmare in practice. The basic idea of their patch is that rsync
! 88: is rewritten to use the new getaddrinfo()/getnameinfo() interface,
! 89: rather than gethostbyname()/gethostbyaddr() as in rsync 2.4.6.
! 90: Systems that don't have the new interface are handled by providing
! 91: our own implementation in lib/, which is selectively linked in.
! 92:
! 93: The problem with this is that it is really hard to get right on
! 94: platforms that have a half-working implementation, so redefining
! 95: these functions clashes with system headers, and leaving them out
! 96: breaks. This affects at least OSF/1, RedHat 5, and Cobalt, which
! 97: are moderately improtant.
! 98:
! 99: Perhaps the simplest solution would be to have two different files
! 100: implementing the same interface, and choose either the new or the
! 101: old API. This is probably necessary for systems that e.g. have
! 102: IPv6, but gethostbyaddr() can't handle it. The Linux manpage claims
! 103: this is currently the case.
! 104:
! 105: In fact, our internal sockets interface (things like
! 106: open_socket_out(), etc) is much narrower than the getaddrinfo()
! 107: interface, and so probably simpler to get right. In addition, the
! 108: old code is known to work well on old machines.
! 109:
! 110: We could drop the rather large lib/getaddrinfo files.
! 111:
! 112: -- --
! 113:
! 114:
! 115: Other IPv6 stuff
! 116:
! 117: Implement suggestions from http://www.kame.net/newsletter/19980604/
! 118: and ftp://ftp.iij.ad.jp/pub/RFC/rfc2553.txt
! 119:
! 120: If a host has multiple addresses, then listen try to connect to all
! 121: in order until we get through. (getaddrinfo may return multiple
! 122: addresses.) This is kind of implemented already.
! 123:
! 124: Possibly also when starting as a server we may need to listen on
! 125: multiple passive addresses. This might be a bit harder, because we
! 126: may need to select on all of them. Hm.
! 127:
! 128: -- --
! 129:
! 130:
! 131: Add ACL support 2001/12/02
! 132:
! 133: Transfer ACLs. Need to think of a standard representation.
! 134: Probably better not to even try to convert between NT and POSIX.
! 135: Possibly can share some code with Samba.
! 136: NOTE: there is a patch that implements this in the "patches" subdir.
! 137:
! 138: -- --
! 139:
! 140:
! 141: proxy authentication 2002/01/23
! 142:
! 143: Allow RSYNC_PROXY to be http://user:pass@proxy.foo:3128/, and do
! 144: HTTP Basic Proxy-Authentication.
! 145:
! 146: Multiple schemes are possible, up to and including the insanity that
! 147: is NTLM, but Basic probably covers most cases.
! 148:
! 149: -- --
! 150:
! 151:
! 152: SOCKS 2002/01/23
! 153:
! 154: Add --with-socks, and then perhaps a command-line option to put them
! 155: on or off. This might be more reliable than LD_PRELOAD hacks.
! 156:
! 157: -- --
! 158:
! 159:
! 160: FAT support
! 161:
! 162: rsync to a FAT partition on a Unix machine doesn't work very well at
! 163: the moment. I think we get errors about invalid filenames and
! 164: perhaps also trying to do atomic renames.
! 165:
! 166: I guess the code to do this is currently #ifdef'd on Windows;
! 167: perhaps we ought to intelligently fall back to it on Unix too.
! 168:
! 169: -- --
! 170:
! 171:
! 172: --diff david.e.sewell 2002/03/15
! 173:
! 174: Allow people to specify the diff command. (Might want to use wdiff,
! 175: gnudiff, etc.)
! 176:
! 177: Just diff the temporary file with the destination file, and delete
! 178: the tmp file rather than moving it into place.
! 179:
! 180: Interaction with --partial.
! 181:
! 182: Security interactions with daemon mode?
! 183:
! 184: -- --
! 185:
! 186:
! 187: Add daemon --no-fork option
! 188:
! 189: Very useful for debugging. Also good when running under a
! 190: daemon-monitoring process that tries to restart the service when the
! 191: parent exits.
! 192:
! 193: -- --
! 194:
! 195:
! 196: Create more granular verbosity 2003/05/15
! 197:
! 198: Control output with the --report option.
! 199:
! 200: The option takes as a single argument (no whitespace) a
! 201: comma delimited lists of keywords.
! 202:
! 203: This would separate debugging from "logging" as well as
! 204: fine grained selection of statistical reporting and what
! 205: actions are logged.
! 206:
! 207: http://lists.samba.org/archive/rsync/2003-May/006059.html
! 208:
! 209: -- --
! 210:
! 211: DOCUMENTATION --------------------------------------------------------
! 212:
! 213:
! 214: Keep list of open issues and todos on the web site
! 215:
! 216: -- --
! 217:
! 218:
! 219: Perhaps redo manual as SGML
! 220:
! 221: The man page is getting rather large, and there is more information
! 222: that ought to be added.
! 223:
! 224: TexInfo source is probably a dying format.
! 225:
! 226: Linuxdoc looks like the most likely contender. I know DocBook is
! 227: favoured by some people, but it's so bloody verbose, even with emacs
! 228: support.
! 229:
! 230: -- --
! 231:
! 232: LOGGING --------------------------------------------------------------
! 233:
! 234:
! 235: Memory accounting
! 236:
! 237: At exit, show how much memory was used for the file list, etc.
! 238:
! 239: Also we do a wierd exponential-growth allocation in flist.c. I'm
! 240: not sure this makes sense with modern mallocs. At any rate it will
! 241: make us allocate a huge amount of memory for large file lists.
! 242:
! 243: -- --
! 244:
! 245:
! 246: Improve error messages
! 247:
! 248: If we hang or get SIGINT, then explain where we were up to. Perhaps
! 249: have a static buffer that contains the current function name, or
! 250: some kind of description of what we were trying to do. This is a
! 251: little easier on people than needing to run strace/truss.
! 252:
! 253: "The dungeon collapses! You are killed." Rather than "unexpected
! 254: eof" give a message that is more detailed if possible and also more
! 255: helpful.
! 256:
! 257: If we get an error writing to a socket, then we should perhaps
! 258: continue trying to read to see if an error message comes across
! 259: explaining why the socket is closed. I'm not sure if this would
! 260: work, but it would certainly make our messages more helpful.
! 261:
! 262: What happens if a directory is missing -x attributes. Do we lose
! 263: our load? (Debian #28416) Probably fixed now, but a test case would
! 264: be good.
! 265:
! 266: -- --
! 267:
! 268:
! 269: Better statistics Rasmus 2002/03/08
! 270:
! 271: <Rasmus>
! 272: hey, how about an rsync option that just gives you the
! 273: summary without the list of files? And perhaps gives
! 274: more information like the number of new files, number
! 275: of changed, deleted, etc. ?
! 276:
! 277: <mbp>
! 278: nice idea there is --stats but at the moment it's very
! 279: tridge-oriented rather than user-friendly it would be
! 280: nice to improve it that would also work well with
! 281: --dryrun
! 282:
! 283: -- --
! 284:
! 285:
! 286: Perhaps flush stdout like syslog
! 287:
! 288: Perhaps flush stdout after each filename, so that people trying to
! 289: monitor progress in a log file can do so more easily. See
! 290: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=48108
! 291:
! 292: -- --
! 293:
! 294:
! 295: Log child death on signal
! 296:
! 297: If a child of the rsync daemon dies with a signal, we should notice
! 298: that when we reap it and log a message.
! 299:
! 300: -- --
! 301:
! 302:
! 303: verbose output David Stein 2001/12/20
! 304:
! 305: At end of transfer, show how many files were or were not transferred
! 306: correctly.
! 307:
! 308: -- --
! 309:
! 310:
! 311: internationalization
! 312:
! 313: Change to using gettext(). Probably need to ship this for platforms
! 314: that don't have it.
! 315:
! 316: Solicit translations.
! 317:
! 318: Does anyone care? Before we bother modifying the code, we ought to
! 319: get the manual translated first, because that's possibly more useful
! 320: and at any rate demonstrates desire.
! 321:
! 322: -- --
! 323:
! 324: DEVELOPMENT --------------------------------------------------------
! 325:
! 326: Handling duplicate names
! 327:
! 328: Some folks would like rsync to be deterministic in how it handles
! 329: duplicate names that come from mering multiple source directories
! 330: into a single destination directory; e.g. the last name wins. We
! 331: could do this by switching our sort algorithm to one that will
! 332: guarantee that the names won't be reordered. Alternately, we could
! 333: assign an ever-increasing number to each item as we insert it into
! 334: the list and then make sure that we leave the largest number when
! 335: cleaning the file list (see clean_flist()). Another solution would
! 336: be to add a hash table, and thus never put any duplicate names into
! 337: the file list (and bump the protocol to handle this).
! 338:
! 339: -- --
! 340:
! 341:
! 342: Use generic zlib 2002/02/25
! 343:
! 344: Perhaps don't use our own zlib.
! 345:
! 346: Advantages:
! 347:
! 348: - will automatically be up to date with bugfixes in zlib
! 349:
! 350: - can leave it out for small rsync on e.g. recovery disks
! 351:
! 352: - can use a shared library
! 353:
! 354: - avoids people breaking rsync by trying to do this themselves and
! 355: messing up
! 356:
! 357: Should we ship zlib for systems that don't have it, or require
! 358: people to install it separately?
! 359:
! 360: Apparently this will make us incompatible with versions of rsync
! 361: that use the patched version of rsync. Probably the simplest way to
! 362: do this is to just disable gzip (with a warning) when talking to old
! 363: versions.
! 364:
! 365: -- --
! 366:
! 367:
! 368: Splint 2002/03/12
! 369:
! 370: Build rsync with SPLINT to try to find security holes. Add
! 371: annotations as necessary. Keep track of the number of warnings
! 372: found initially, and see how many of them are real bugs, or real
! 373: security bugs. Knowing the percentage of likely hits would be
! 374: really interesting for other projects.
! 375:
! 376: -- --
! 377:
! 378: PERFORMANCE ----------------------------------------------------------
! 379:
! 380: Allow skipping MD4 file_sum 2002/04/08
! 381:
! 382: If we're doing a local transfer, or using -W, then perhaps don't
! 383: send the file checksum. If we're doing a local transfer, then
! 384: calculating MD4 checksums uses 90% of CPU and is unlikely to be
! 385: useful.
! 386:
! 387: We should not allow it to be disabled separately from -W, though
! 388: as it is the only thing that lets us know when the rsync algorithm
! 389: got out of sync and messed the file up (i.e. if the basis file
! 390: changed between checksum generation and reception).
! 391:
! 392: -- --
! 393:
! 394:
! 395: Accelerate MD4
! 396:
! 397: Perhaps borrow an assembler MD4 from someone?
! 398:
! 399: Make sure we call MD4 with properly-sized blocks whenever possible
! 400: to avoid copying into the residue region?
! 401:
! 402: -- --
! 403:
! 404: TESTING --------------------------------------------------------------
! 405:
! 406: Torture test
! 407:
! 408: Something that just keeps running rsync continuously over a data set
! 409: likely to generate problems.
! 410:
! 411: -- --
! 412:
! 413:
! 414: Cross-test versions 2001/08/22
! 415:
! 416: Part of the regression suite should be making sure that we
! 417: don't break backwards compatibility: old clients vs new
! 418: servers and so on. Ideally we would test both up and down
! 419: from the current release to all old versions.
! 420:
! 421: Run current rsync versions against significant past releases.
! 422:
! 423: We might need to omit broken old versions, or versions in which
! 424: particular functionality is broken
! 425:
! 426: It might be sufficient to test downloads from well-known public
! 427: rsync servers running different versions of rsync. This will give
! 428: some testing and also be the most common case for having different
! 429: versions and not being able to upgrade.
! 430:
! 431: The new --protocol option may help in this.
! 432:
! 433: -- --
! 434:
! 435:
! 436: Test on kernel source
! 437:
! 438: Download all versions of kernel; unpack, sync between them. Also
! 439: sync between uncompressed tarballs. Compare directories after
! 440: transfer.
! 441:
! 442: Use local mode; ssh; daemon; --whole-file and --no-whole-file.
! 443:
! 444: Use awk to pull out the 'speedup' number for each transfer. Make
! 445: sure it is >= x.
! 446:
! 447: -- --
! 448:
! 449:
! 450: Test large files
! 451:
! 452: Sparse and non-sparse
! 453:
! 454: -- --
! 455:
! 456:
! 457: Create mutator program for testing
! 458:
! 459: Insert bytes, delete bytes, swap blocks, ...
! 460:
! 461: -- --
! 462:
! 463:
! 464: Create configure option to enable dangerous tests
! 465:
! 466: -- --
! 467:
! 468:
! 469: Create pipe program for testing
! 470:
! 471: Create pipe program that makes slow/jerky connections for
! 472: testing Versions of read() and write() that corrupt the
! 473: stream, or abruptly fail
! 474:
! 475: -- --
! 476:
! 477:
! 478: Create test makefile target for some tests
! 479:
! 480: Separate makefile target to run rough tests -- or perhaps
! 481: just run them every time?
! 482:
! 483: -- --
! 484:
! 485: RELATED PROJECTS -----------------------------------------------------
! 486:
! 487: rsyncsh
! 488:
! 489: Write a small emulation of interactive ftp as a Pythonn program
! 490: that calls rsync. Commands such as "cd", "ls", "ls *.c" etc map
! 491: fairly directly into rsync commands: it just needs to remember the
! 492: current host, directory and so on. We can probably even do
! 493: completion of remote filenames.
! 494:
! 495: -- --
! 496:
! 497:
! 498: http://rsync.samba.org/rsync-and-debian/
! 499:
! 500:
! 501: -- --
! 502:
! 503:
! 504: rsyncable gzip patch
! 505:
! 506: Exhaustive, tortuous testing
! 507:
! 508: Cleanups?
! 509:
! 510: -- --
! 511:
! 512:
! 513: rsyncsplit as alternative to real integration with gzip?
! 514:
! 515: -- --
! 516:
! 517:
! 518: reverse rsync over HTTP Range
! 519:
! 520: Goswin Brederlow suggested this on Debian; I think tridge and I
! 521: talked about it previous in relation to rproxy.
! 522:
! 523: Addendum: It looks like someone is working on a version of this:
! 524:
! 525: http://zsync.moria.org.uk/
! 526:
! 527: -- --
! 528:
FreeBSD-CVSweb <freebsd-cvsweb@FreeBSD.org>
![[BACK]](/icons/cvsweb/back.gif){kind=icon}
Return to TODO CVS log ![[TXT]](/icons/cvsweb/text.gif){kind=icon}
![[DIR]](/icons/cvsweb/dir.gif){kind=icon}
Up to [ELWIX - Embedded LightWeight unIX -] / embedaddon / rsync