Reliability
Zero Message Loss – TCP only isn’t enough!
Application Level Acknowledgement via Reliable Log Transfer Protocol
- syslog-ng Premium Edition can send and receive log messages in a reliable way over the TCP transport layer using the Reliable Log Transfer Protocol™ (RLTP™). RLTP™ is a new transport protocol that prevents message loss during connection breaks. It detects the last received message on the receiving end and then starts resending messages from that point, ensuring messages are not duplicated at the receiving end in case of a connection break.
Disk-based Message Buffering
- The Premium Edition of syslog-ng stores messages on the local hard disk if the central log server or the network connection becomes unavailable. The syslog-ng application automatically sends the stored messages to the server when the connection is reestablished, in the same order the messages were received. The disk buffer is persistent – no messages are lost even if syslog-ng is restarted.
Flow Control
- Flow-control uses a control window to determine if there is free space in the output buffer of syslog-ng for new messages. If the output buffer is full, then the destination cannot accept new messages for some reason: for example, it is overloaded, or the network connection became unavailable. In such cases, syslog-ng stops reading messages from the source until some messages have been successfully sent to the destination.
Professional Support
- Major releases of syslog-ng PE are supported and maintained for a long time, as described in the BalaBit version policy.
Scalability
Extreme Message Rate Collection
- The syslog-ng application is optimized for performance, and can handle enormous amount of messages. Depending on its exact configuration, it has been known to process over 650,000 messages per second real-time, and over 24 GB raw logs per hour on standard server hardware.
Collection from Thousands of Log Sources
- With the syslong-ng client-relay architecture, IT organizations can collect log messages from more than 10,000 log sources across a geographically distributed environment on one central log server.
Security
Secure Transfer using SSL/TLS
Log messages may contain sensitive information that should not be accessed by third parties. Therefore, syslog-ng Premium Edition uses the Transport Layer Security (TLS) protocol to encrypt the communication. TLS also allows the mutual authentication of the host and the server using X.509 certificates.
Secure, Encrypted Log Storage
The Premium Edition of syslog-ng can store log messages securely in encrypted, compressed, indexed, and timestamped binary files, so any sensitive data is available only for authorized personnel who have the appropriate encryption key. Timestamps can be requested from external Timestamping Authorities.
Flexibility
Support for more than 50 Server Platforms
The syslog-ng Premium Edition application supports several architectures, including x86, x86_64, and SUN SPARC on a variety of operating systems.
Collect from a Wide Variety of Sources
syslog-ng Premium Edition can natively collect and process log messages from SQL databases enabling users to easily manage log messages from a wide variety of enterprise software and custom applications.
Windows Support
The syslog-ng Premium Edition version 5 LTS offers complete support for Windows platforms. You can install the syslog-ng Premium Edition application on Windows operating systems as a client or central logserver or install the lightweight syslog-ng Agent for Windows.
Read Log Messages from Any Text File
Some applications use many different log files, and sometimes these files are not even located in the same folder. Automatically generated file and folder names are also often a problem. To solve these issues, the filenames and paths specifying the log files read by syslog-ng can include wildcards, and syslog-ng can automatically scan entire subfolder-trees for the specified files. The syslog-ng Premium Edition application is also able to process multi-line log messages, for example, Apache Tomcat messages.
Filter, Parse, Re-Write
The syslog-ng application can sort the incoming log messages based on their content and various parameters like the source host, application, and priority. Directories, files, and database tables can be created dynamically using macros. Complex filtering using regular expressions and boolean operators offers almost unlimited flexibility to forward only the important log messages to the selected destinations.
Normalize data with PatternDB
The syslog-ng application can compare the contents of the log messages to a database of predefined message patterns.
- Real-time log message classification
By comparing log messages to known patterns, syslog-ng is able to identify the exact type of the messages, and sort them into message classes. The message classes can be used to classify the type of the event described in the log message. The message classes can be customized, and for example can label the messages as user login, application crash, file transfer, etc. events.
- Extracting important information from messages
In addition to classifying messages, you can also add different tags which can be used later for filtering messages, for example, to collect messages tagged as user_login to a separate file or to perform conditional post processing on the tagged messages.
- Real-time event correlation
syslog-ng also makes real time event correlation possible. This can be useful in many different situations. For example important data for a single event is often scattered into multiple syslog messages. Also login and logout events are often logged far away from each other, even in different log files, making log analysis difficult. Using correlation these can be collected into a single new message.
Ref: Terreactive
