Snowflake algorithm ID generation”.
http://blog.csdn.net/w200221626/article/details/52064976
There are no rules for ID generated by UUID or GUID.
SnowflakeThe algorithm is implemented by Twitter engineers in order to achieve incremental but not repeated ID.
The top 41 are said to be able to support 1023 machines by 2088, the top 10 can support 1023 machines, and the last 12-bit serial number can generate 4095 self-increasing IDs in a millisecond.
There are many ways of primary keys in data: database increment and program generation. Snowflake algorithm is commonly used in program generation. This algorithm has many explanations on the Internet. There is not much explanation here.
The generated ID has roughly the following components:
SnowflakeEach ID generated by the algorithm is a 64-bit integer, and its core algorithm is relatively simple and efficient, the structure is as follows:
41The time series of bits are accurate to milliseconds, and the length of 41 bits can be used for 69 years. There is also a very important role in time ranking, which can be sorted according to time.
5The data center identifier of the bit, the 5 bit length supports the deployment of 32 nodes at most.
5The 8 bit length supports the deployment of 255 nodes.
12Bit count sequence number, which is a series of self-increasing ids, can support the same node to generate multiple ID numbers in the same millisecond, and 12-bit count sequence number can support each node to generate 4095 ID numbers per millisecond.
The highest bit is the symbol bit, always 0, and is not available.
According to the generation rules and actual code:
（Detailed algorithm: https://segmentfault.com/a/1190000011282426#articleHeader2)
In multithreading, use lock.
C# Implementation of Snowflake algorithm
/// <summary>
/// Dynamic Production Regular ID Snowflake Algorithms are implemented by Twitter engineers for incremental, non-repetitive IDs/ / / http://blog.csdn.net/w200221626/article/deTails/52064976/ / / C# implementation of Snowflake algorithm/ / / < /summary>Public class Snowflake{Private static long machineId; / / machine IDPrivate static long datacenterId = 0L; / / dataIDPrivate static long sequence = 0L; / / counting starts from zero.Private static long twepoch = 687888001020L; / / unique time random quantityPrivate static long machineIdBits = 5L; / / machine code bytesPrivate static LOng datacenterIdBits = 5L; / / data bytesPublic static long maxMachineId = -1L ^ -1L < < (int)MachineIdBits; / / max machine IDPrivate static long maxDatacenterId = -1L ^ (-1L < < (int) DatacEnterIdBits); / / max data IDPrivate static long sequenceBits = 12L; / / counter bytes, 12 bytes used to save count numbers.Private static long machineIdShift = sequenceBits; the number of left shifts in the // machine code data is the number of digits occupied by the subsequent counterPrivateStatic long datacenterIdShift = sequenceBits + machineIdBits;Private static long timestampLeftShift = sequenceBits + machineIdBits + datacenterIdBits; the // timestamp left move bit is machine code + counter total bytes + data bytesPPublic static long sequenceMask = 1 L ^ - 1 L & lt; & lt; int) sequenceBits; counts can be generated in a microsecond // and wait until the next subtlety is reachedgeneratePrivate static long lastTimestamp = -1L; / / last timestampPrivate static object syncRoot =New object (); / / lock objectStatic Snowflake snowflake;Public static Snowflake Instance (){If (snowflake = = null)Snowflake = new Snowflake ();Return snowflake;}Public Snowflake (){Snowflakes (0L, -1);}Public Snowflake (long machineId){Snowflakes (machineId, -1);}Public Snowflake (long machineId, long datacenterId){SnowfLakes (machineId, datacenterId);}Private void Snowflakes (long machineId, long dataCenterId){If (machineId > = 0){If (machineId ≫ maxMachineId){Throw new Exception ("machine code ID illegal");}Snowflake.machineId = machineId;}If (datacenterId &g)T; = 0){If (datacenterId > maxDatacenterId){Throw new Exception ("data center ID illegal");}Snowflake.datacenteRId = datacenterId;}}/ / / < summary>/ / / generate the current timestamp./ / / < /summary>/ / / < returns> millisecond < /returns>Private static long GetTImestamp (){/ let him start in 2000.Return (long) (DateTime.UtcNow - New DateTim)E (2000, 1, 1, 0, 0, 0, DateTimeKind.Utc).TotalMilliseconds;}/ / / < summary>/ / / get the next microsecond timestamp./ / / < /summary>/ / / < param name= "lastTimestamp" > </param>/ / / < returns> < /returns>Private static long GetNextTimestamp(long lastTimestamp){Long timestamp = GetTimestamp ();Int count =0;While (timestamp & lt; = lastTimestamp) / / There may be errors in getting the new time, which is as strict as comb in requiring machine time{Count++;If (count > 10)Throw new ExcepTion ("machine time may be wrong");Thread.Sleep (1);Timestamp = GetTimestamp ();}Return timestamp;}/ / / < summary>/ / / get long plastic ID/ / / < /summary>/ / / < returns> < /returns>Public long GetId (){Lock (syncRoot){Long timestamp = GetTimestamp ();If (Snowflake.lastTimestamp = = timestamp){/ / generate ID in the same subtlety.Sequence = sequence + 1 & amp; sequenceMask; // with & amp; calculates whether the count generated within that microsecond has reached the upper limitIf (sequence = = 0){/ / a subtly generated ID count has reached the upper limit, waiting for the next subtlety.Timestamp = GetNextTimestamp (Snowflake.lastTimestamp);}}Else{/ / different microseconds generate IDSequence = 0L;}If (timestamp < lastTimestamp){Throw new Exception ("time stamp is smaller than the last timestamp when ID is generated, so it is abnormal").}Snowflake.lastTimestamp = timestamp; / / the current timestamp is saved as the last timestamp for generating ID.Long Id = ((timesta)MP - twepoch) < < (int) timestampLeftShift)(datacenterId < < (int) DaTacenterIdShift)(machineId < < (int) machineIdShift)Sequence;Return Id;}}}Copy codeCopy code[TestClass]Public class SnowflakeUnitTest1{/ / / < summary>/ / dynamic production regular ID Snowflake algorithmIt is Twitter's engineer who implements ID instead of repeating it./ / / < /summary>[TestMethod]Public void SnOwflakeTestMethod1 (){Var IDS = new List< long> ();For (int i)= 0; I < 1000000; i++) / / test at the same time 100W ordered ID{Ids.Add (Snowflake.Instance ().G)EtId ());}For (int i = 0; I < ids.Count - 1; i++){Assert.IsTrue (ids[i] < ids[i+1]);}}}Namespace ConsoleApplicaTionTester{Class Program{Static void Main (string[] args){FOr (int i = 0; I < 1000; i++){Console.WriteLine ("start execution" + DateTime.)Now.ToString ("yyyy-MM-dd HH:mm:ss:ffffff") + "" + Snowflake.Instance ().GetId ();Console.WriteLine ("Snowflake.maxMachineId:" + Snowflake.maxMachineId);}}}}http://www.infoq.com/cn/news/2017/09/snowflake
Researchers from Microsoft Research Institute, Cambridge University, and Princeton University have built a branch of. NET that implements the addition of an API that supports manual memory management at runtime. The details of the research method and the performance improvement were published in the name “Project S”.Now flake: Paper on Non-blocking Safe Manual Memory Management in. NET (“Snowflake” Project: Non-blocking, Safe. NET Manual Memory Management).
“The Snowflake project aims to implement manual memory management in. NET, an improvement that is considered very useful for some applications. Modern programming languages such as C#,.NET all use garbage collection mechanisms to free programmers from the task of managing objects. The advantages of this mechanism are well known and involveImprove productivity, improve program stability, memory security and prevent malicious operation. But garbage collection mechanisms have some performance costs, which are not easily noticed in many cases, but are problematic in some cases. “Snowflake” project researchers pointed out that for theManual memory management benefits from running data analysis and streaming tasks on systems with hundreds of GB of heap memory.
“The manual memory management introduced by the Snowflake project works in parallel with the garbage collection mechanism. Developers usually use the garbage collection mechanism, but they can also choose the manual memory management when the environment needs it. The improvements introduced into runtime will not affect existing applications, and will improve multithreading.Application performance. The Snowflake project implements “assigning and releasing independent objects anywhere in the program, and ensuring that manually managed objects also enjoy full type and timing security, even when concurrent access exists.”
“Two new concepts, object Owner and Shield, are introduced in Snowflake. They are implemented as CoreCLR and CorefX level APIs. The “owner” represents a location in the stack or heap, which preserves only the allocated objects in the manual heap.A reference. The owner gets it from the shield, and the shield is introduced to avoid deallocating manual objects when they are accessed by multiple threads. The shield ensures that the object is not removed from the heap until it is redistributed using the thread of an object. The paper is likeThe following mechanisms are elaborated:
Our solution… Inspired by the concept of Hazard Pointer in unlocked data structure research. We introduced a mechanism that when a thread wants to access a manual object through one of the “owner” locations, the intent is to publish the thread bookIn the ground state (TLS, Thread-Local State). This registration process can be seen as creating a “shield” that protects the object from redistribution and authorizes the thread that publishes the registration to access the object directly, such as calling the object’s method, or transforming (mu).Tate) the field of the object. At the same time, no thread (the same thread or another thread) is allowed to reallocate objects and reclaim the memory of objects. Once the client code no longer needs to access the object, the dispose “shield” can be released from the object’s TLSRemoved the reference to the object. Direct access to objects retrieved from the shield is an unsafe operation because the actual redistribution operation is still allowed to continue after the shield is released.
A series of test results for a given scenario are presented, which show that the performance of the Snowflake project is better than that of the garbage collection mechanism. Among them, “the peak set of work achieved as high as three times the performance improvement, at runtime achieved twice the performance improvement.” The test results give good results.It can be improved. This is because garbage collection takes a lot of time traversing the object graph to free memory when the object pool is very large.
MicrosoftThere is no detailed plan to include snowflake projects in.NET. But considering that this is a non-intrusive and secure mechanism, we hope to integrate similar functionality in future versions of. NET.
Microsoft Explores Manual Memory Management in.NET with Snowflake