How many threads can i create

Connect and share knowledge within a single location that is structured and easy to search. I have a python program that uses the threading module and runs multithreads that executes a function.

The number 8 below is the number of threads in the program but I need to know what is the number of threads my device can handle to run the program as fast as possible. Can you tell me what is the number of threads my program can handle without causing issues?

And how is this computed please? The iHQ is a 4 core processor. This means your processor can support 8 threads. Sign up to join this community. The best answers are voted up and rise to the top. Stack Overflow for Teams — Collaborate and share knowledge with a private group.

Create a free Team What is Teams? Learn more. How many threads can I run in my program? Ask Question. Asked 2 years, 8 months ago. Active 2 years, 8 months ago. Viewed 4k times. Improve this question.

Please do not reply with a comment. You need to tell us about each aspect of your program. For example, if each thread uses 3g of RAM and you have 8gb your practical limit is 2. And threads shouldn't cost any execution time while semaphore-blocked, they should sit in the blocked queue until the semaphore is released.

Show 1 more comment. Max Virtual memory that a given OS variant supports. Linux Kernel 2. For 2. What kindda kernel scheduler you have. Comparing Linux 2. So also the SMP Capabilities of the kernel schedule also play a good role in max number of sustainable threads in a system. Jay D Jay D 3, 3 3 gold badges 29 29 silver badges 47 47 bronze badges. Note that these virtual memory limits only apply to bit systems.

On 64 bits you won't run out of virtual memory. Add a comment. Andrew Grant Andrew Grant Yes, and it should be used in conjunction with a queue or pool of requests. Andrew: Why? It should add a task to the thread pool each time it receives a request. It is up to the thread pool to allocate a thread for the task when there is one available. So what do you do when you have hundreds of requests coming in and are out of threads?

Create more? Return an error? Place your requests in a pool that can be as large as need be, and then feed these queued requests to your thread pool as threads become free. Typically, there are many more tasks than threads. As soon as a thread completes its task, it will request the next task from the queue until all tasks have been completed. Andrew: I am not sure what python thread pool the OP is using, but if you want a real world example of this functionality I am describing: msdn.

Chad Okere Chad Okere 4, 1 1 gold badge 19 19 silver badges 19 19 bronze badges. After reading this, I tried running sieve of Eratosthenes tasks on three threads.

Thanks for the heads up. Next, I'll try a scenario that involves some database calls. There are two at least types of tasks: CPU bound e. If your tasks are CPU bound then you should consider multiprocessing instead of multithreading. Of course, it depends on many things, that's why you must measure yourself. That's a tad higher than I would have expected as well. Still, if that's what you got, then that's what you got, I can't argue with that.

For this specific application, most threads are just waiting a response from the DNS server. So, the more parallelism, the better, in wall-clock time. I speak from experience here. Matthew Lund Matthew Lund 3, 6 6 gold badges 27 27 silver badges 37 37 bronze badges. Can you mention some of the numbers you've seen for thread count? It'd be helpful to just get a sense of it. Hot Licks Hot Licks The whole point of threads was before multicore and multiple processors became prevalent is to be able to mimic having multiple processors on a machine that has just one.

That's how you get responsive user interfaces-- a main thread and ancillary threads. The statement I made was that the number of cores on a machine represents a hard limit on the number of threads that can be doing work at a given time, which is a fact.

Anyway - you have GIL in Python, which makes threads only theoretically parallel. No more than 1 thread can run simultaneously, so it's only the responsiveness and blocking operations that matters. Rich B: A thread pool is just one of many ways to handle a collection of threads.

It is a good one, but certainly not the only one. Show 9 more comments. For Python that's especially true, as multiple processes can run in parallel, while multiple threads - don't. The cost is however quite high. You have to start new Python interpreter each time, and connect to DB with each process or use some pipes redirection, but it also comes at a price.

Switching between processes is - most of the time - more expensive than switching between threads whole context switch instead of some registers.

At the end it depends heavily on your threading-lib. As the questions revolved around threading, I assume that processes are out of question already. Fair enough. I'm not sure why that's why I'm getting a -2 ding to the score, though, unless people really want to see thread-only answers, rather than including other answers that work. Process creation can be done once at startup ie, a process pool instead of a thread pool.

Amortized over the application duration, this may be small. Adding to your thread count should not randomly crash your app. There's some reason. You would do well to figure out the cause because it may effect you even with fewer threads in some circumstances, who knows. How do I limit the number of threads in python? How many threads is too many Python?

How many threads can be executed at a time Python? How many threads can run in parallel in Python? How many threads should I create? Is multiprocessing faster than multithreading? Can Python run in parallel? Are Python threads real? Is NumPy thread safe? Is Python set thread safe? How many threads can I run? What does 4 cores and 4 threads mean? How many maximum threads can you create?

How many threads can Windows handle? How many threads can Windows 10 handle? How much RAM should be in a thread? How do I increase thread limit in Windows?