Server size :

Single node deployment , disk 1T, Memory 128G

Concurrent import scale :

1, Multithread concurrent import csv file

2,csv File Division 1 ten thousand 、10 ten thousand 、100 ten thousand 、200 Ten thousand records 4 Kinds of size

3, Every csv Corresponding to one collection

Concurrent query size

1, Multithreading concurrent queries are different collection

2, There are two scenarios: full table query and local query

Performance test results :

Import performance

csv file size ( Ten thousand records ) Number of concurrent threads Import time ( second )   Cumulative import csv Number of documents
200 1 60 5000
200 10 105 5000
200 40 330( Peak memory 110G) 5000
100 1 20 10000
100 10 32 10000
100 100 203( Peak memory 80G) 10000
100 200 478( Peak memory 120G) 10000
10 1 4 100000
10 10 6 100000
10 100 39 100000
10 500 370( Peak memory 95G) 100000
1 1 1 1000000
1 10 1 1000000
1 100 8 1000000
1 500 32 1000000

Query performance

collection size ( Ten thousand records ) Number of concurrent threads Full table query takes time ( second )   Some queries are time consuming —— check 10 Ten thousand records ( second )
200 1 5.5 4
200 10 23 6.2
200 40 126 13.7
100 1 1.3 2.4
100 10 8.9 3.7
100 20 11 4
100 40 70 6.2
10 1 0.6 0.5
10 10 0.6 0.5
10 100 5 1.3
10 500 85 5
1 1 0.6 /
1 10 0.4 /
1 100 0.7 /
1 500 2 /

MongoDB Big data high concurrent read and write performance test report more related articles

  1. Distributed big data is highly concurrent web Development framework

    One . introduction Usually we think of static web pages as html Our website is the fastest , But since the advent of dynamic web pages , A lot of interactive data comes from the database , Data is also constantly changing , In addition to some news and information websites , Use html Static to improve access speed is not very ...

  2. Big data high concurrency system architecture practical solution (LVS Load balancing 、Nginx、 Shared memory 、 Huge amounts of data 、 Queue cache )

    Course introduction : With the development of the Internet , High concurrency . Websites with large amounts of data are demanding more and more . And these high requirements are the combination of basic technology and details . This course starts from the actual case to give you the original scene to reproduce the common technical points and detailed drills of high concurrency architecture . Through the study of this course , universal ...

  3. php How to deal with big data with high concurrency

    Big data solutions Use the cache : Usage mode :1, Use the program to save directly to memory . The main use of Map, especially ConcurrentHashMap. Using the caching framework . Common framework :Ehcache,Memcache,Redis etc. . ...

  4. 15 set java Architects 、 colony 、 High availability 、 Highly scalable 、 High performance 、 High concurrency 、 performance optimization 、Spring boot、Redis、ActiveMQ、Nginx、Mycat、Netty、Jvm Large distributed project practice video tutorial

    * { font-family: "Microsoft YaHei" !important } h1 { color: #FF0 } 15 set java Architects . colony . High availability . Highly scalable . ...

  5. 15 set java Architects 、 colony 、 High availability 、 Highly expandable exhibition 、 High performance 、 High concurrency 、 performance optimization Redis、ActiveMQ、Nginx、Mycat、Netty、Jvm Large distributed project practice video tutorial

    * { font-family: "Microsoft YaHei" !important } h1 { color: #FF0 } 15 set java Architects . colony . High availability . Highly expandable exhibition ...

  6. Qunar The air ticket Technology Department has a key indicator for the whole year : Search cache hit rate , It was done at the time >99.7%. Further back , Every increase 0.1%, The difficulty of optimization has increased exponentially . Even one in a thousand , It also directly affects the user experience , It affects the sales of tens of thousands of air tickets every day . In high concurrency scenarios , Provides thread safe objects 、 Method . Like the classic ConcurrentHashMap, It's better than HashMap, There are smaller granularity locks , Concurrent read and write performance is better . Thread safe StringBuilder replace S

    Qunar The air ticket Technology Department has a key indicator for the whole year : Search cache hit rate , It was done at the time >99.7%. Further back , Every increase 0.1%, The difficulty of optimization has increased exponentially . Even one in a thousand , It also directly affects the user experience , It affects tens of thousands of machines every day ...

  7. php Interview question two -- The solution to the problem of large flow and high concurrency of website ( from url To the hard disk to solve high concurrency solution summary )

    php Interview question two -- The solution to the problem of large flow and high concurrency of website ( from url To the hard disk to solve high concurrency solution summary ) One . summary Solve the problem of large traffic and high concurrency from outside to inside --- From submitting a url Start ( Start with the user pressing the enter button in the search bar ) url In the beginning, it will be d ...

  8. mongodb Special topic test on high concurrent update performance of three storage engines

    The background that Recently, the feedback from Beijing financial channel is used to store real-time data of stock market MongoDB Database write response requests are slow , It is difficult to keep up with the level of service write speed . We analyzed the online scene , Found last year upgraded to SSD After the disk , Disk for data persistence IO Spending is no longer ...

  9. mysql High concurrency of big data

    One . The design of database structure If you can't design a reasonable database model , It will not only increase the difficulty of programming and maintenance of client and server programs , And it will affect the actual performance of the system . therefore , Before a system is implemented , It is necessary to design a complete database model . ...

Random recommendation

  1. Mobile solutions fixed and input Get the focus of soft keyboard pop-up problems that affect positioning

    Scene description , When document Not high enough window At the height of , If in ip6 The height of the document in is smaller than that of the form , The area designed at the bottom is not at the bottom of the form , Just leave a blank space, as shown in the gray section below 1. Solve initialization document height , ...

  2. iScroll The drop-down refresh

    <!doctype html> <html> <head> <meta charset="utf-8"> <script ty ...

  3. bzoj 1189

    subject :http://www.lydsy.com/JudgeOnline/problem.php?id=1189 This topic seems to use very high-end network flow . There is one thing special about this problem , There are people in every open space , No ...

  4. [ turn ] Linux Asynchronous I/O Explained

    Linux Asynchronous I/O Explained (Last updated: 13 Apr 2012) *************************************** ...

  5. C++ Of ch1&amp;ch2 Sort it out

    C++: Tearful C, Semantics C I didn't learn the language well , In learning this course, I will shed tears of ignorance ( Just for me ). The development of computer programming language : machine language [ Binary instructions that a computer can recognize ]——> assembly language [ Translate machine instructions into ...

  6. web front end (7)—— understand CSS style , introduce css The way of pattern

    CSS In front of the general introduction of css, Start with this blog post , I will introduce it in detail , At the beginning of the introduction web Front end , Open Baidu home page , I asked a question at the time , Why baidu home input box can be placed in the middle , Because of css The control of , We can play ...

  7. Visual Studio 2015 compile Lua 5.3.4 Pit encountered

    The pit is not shallow , Encounters an error :" LNK1561: The entry point has to be defined ", Solution delete rebuild , Step by step , Or an error . As shown in the figure below : First , It has to be changed to DLL perhaps LIB( dynamic / Static library ), If it should be ...

  8. Ansible--inventory

    brief introduction Inventory yes Ansible A configuration file that manages the host information , It's equivalent to a system HOSTS File functionality , Default store in /etc/ansible/hosts. For the convenience of batch management host , Easy to use the host grouping , ...

  9. Linux Tuning program , Modify the maximum number of connections (ulimit command )【 turn 】

    from :http://blog.csdn.net/liangxiaozhang/article/details/8363435 Linux For each user , The system limits its maximum number of processes . To improve performance , According to the device resources ...

  10. Spring RedisTemplate operation - The hash operation (3)

    @Autowired @Resource(name="redisTemplate") private RedisTemplate<String, String> rt; ...