.NET Core uses NLog to implement log collection through Kafka.

First, preface
NET CoreIt’s getting more popular because it has the power of the original. NET Framework running on multiple platforms. Kafka is rapidly becoming the standard messaging technology in the software industry. This article briefly introduces how to use.NET (Core) and Kafka to implement NL.Target of OG.

In the daily project development process, Spring Boot + Logback in the Java system is easy to access Kafka to achieve log collection, in. NET and. NET Core has been accustomed to using NLog as a log component. In order to make the micro service ringIn the context of dotnet and Java services are unified log collection, the next article will introduce the unified access mode of the two languages. The goal of writing this component is to get team members to write JsonLayout without NLog in the same format as the Java service outputTo Kafka, it is difficult to simplify the configuration of developers. Of course, the cost is not flexible.

Two, open source
By implementing NLog’s Target, access to Kafka transfers logs to the components of Logstash.

https://github.com/maxzhang1985/NLog.Kafka

Three. Use
Building a project
NLog.KafkaComponents support. NET 4.5 + and NET Standard 1.6 + where they can be used in traditional. NET and, of course, cross-platform use of. NET Core (Win, Linux, Mac).

Project reference
NLog 4.5.8
NLog.Kafka
librdkafka.redist
The reference to librdkafka. redist is due to the use of the dependency library Confluent. Kafka 0.11.5, Confluent. Kafka uses the famous librdkafka open source library, which is written in C++ asOther languages, such as C + +, C, Python and Node, are the foundation of Kafka drivers.

To configure
NLog.config is set up in the project and is set to Copy always. The contents are as follows:

Writing test code
class Program
{
static void Main(string[] args)
{

        Logger logger = LogManager.GetCurrentClassLogger();

        MappedDiagnosticsContext.Set("item1", "haha");
        for(int i = 0; i < 10; i++)
        {
            logger.Info("hello world");
            Console.WriteLine("sended");
        }

        Console.ReadKey();
    }

}
LogstashTo configure
input {
kafka {
bootstrap_servers => “127.0.0.1:9092”
group_id => “logstash”
topics => “loges”
codec => “json”
}
}

output{
elasticsearch {
hosts => [“127.0.0.1:9002”]
index => “log_{[appname]}_%{+YYYY.MM.dd}”

}
#stdout { codec => rubydebug }
}
Four. Finally
The attached Demo and open source library address: https://github.com/maxzhang1985/NLog.Kafka

GitHub：https://github.com/maxzhang1985/YOYOFx If you feel that you can also invite Star, welcome to communicate with each other.