Initial Plugin Design
Plugins: Netlogger plugin (see netlogger section for more info)
- Purpose: send performance and restart information to netlogger for archiving, and so the GUI can display performance info.
- parameters to plugin
- netlogger URL
- "extra" netlogger variable=value pairs
- transfer id
- on plugin initialization
- create netlogger. streams will be cached among multiple plugin instances with the same output parameters. netlogger message #1 and netlogger message #2 will sent on separate streams.
- associate transfer-id with operation.
- Detect performance messages on control channel
- For each message received, forward (with all-transfer aggregate information) to netlogger (message type #1 on netlogger stream #1).
- Detect restart-related events on control channel, sending type #2 netlogger message on netlogger stream #2.
Remote control plugin (see Remote Control Protocol for more info)
- Purpose: allow remote control of the ftp client library to trigger restart events and tune the number of data connections.
- parameters to plugin
- transfer id
- timeout for multiple-clicks
- (optional ldap url to store port # into)
- on plugin initialization
- listen on port
- publish port number (printf or ldap)
- on connect to port
- read message
- close
- Protocol Messages:
- "+parallelism <transfer-id>"
- Increment the parallelism level for the transfer
- Reset parallelism-update timer
- "-parallelism <transfer-id>"
- Decrement the parallelism level for the transfer
- Reset parallelism-update timer
- "fault <transfer-id>"
- Timer events
- parallelism-update timer expires (not cancelled)
- restart transfer, with new parallelism level
- fault-update timer (not cancelled)
- restart transfer, with the current parallelism level
Reliability plugin
- Purpose: restart transfers (optionally switching between equivalent data sources) automatically, when low-performance threshhold is exceeded for too long, or when an error occurs. Persistent state is stored on disk so that the application can be restarted without confusing the GUI.
- Parameters to plugin:
- persistent state file directory
- transfer id
- equivalent source URLs (for automatic switch over)
- MinTransferRate
- low-water mark for transfer throughput
- MinTransferTimeOut
- IdleTimeout
- Amount of time to wait until the server sends any message on the control channel.
- RestartDelay
- amount of time to wait before restarting a transfer
- TriesBeforeAlternate
- number of times a transfer may be restarted before trying the next alternate.
- Monitor performance marker responses
- Restart IdleTimer
- Compare file transfer performance to low-water mark, if better, cancel MinTransferTimer, if worse and MinTransferTimer isn't running, start MinTransferTimer
- If MinTransferTimer expires, do restart
- Monitor restart marker responses
- Restart IdleTimer
- Update state file's restart marker.
- All messages to server
- All server replies
- If IdleTimer expires, do restart
- Restart: cancel all timers, increment RestartCounter. If RestartCounter exceeds TriesBeforeAlternate, choose different server if possible. Register restart to happen after RestartDelay.