summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2012-12-21Percpu tag allocatorpercpu-tag-allocatorKent Overstreet
2012-10-04mtip32xx: kill hctxJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04mtip32xx: multiqueueJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04null: add support for multiple devices (nr_devices option)Jens Axboe
Use 2 by default. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04blk-mq: fix bug in requeue list batch addingJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04null: use llist_del_all()Jens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04null: add bio queue methodJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04block: cleanup IO accounting interfaceJens Axboe
Use a sane prefixed name for all the related functions, and put them close together in the blk-core.c file. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04blk-mq: add drive accountingJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04blk-mq: add auto-unplug for deeper depthsJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04blk-mq: ensure that we move all entries if a CPU goes downJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04blk-mq: use llist_del_all() in IPI completionJens Axboe
Saves us a cmpxchg for each list entry. Just yank all entries off the list and iterate them, loop around and check again later when done. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04blk-mq: move ctx queue/lock to separate smp cachelineJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04blk-mq: remove struct blk_mq_ctx from global scopeJens Axboe
It's an internal structure, lets not expose details of it to drivers. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04blk-mq: kill hctx from blk_mq_end_io()Jens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04blk-mq: cleanup some preemption messJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04blk-mq: more mem init fixesJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04blk-mq: handle allocation failures during queue initJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04blk-mq: get rid of passing in a lock to blk_mq_init_queue()Jens Axboe
It was a disaster on the API side for blk_init_queue(), and in the current code we don't even use the functionality anymore. So lets get rid of this, along with the BLK_MQ_F_SHOULD_LOCK flag. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04blk-mq: style and error path fixupsJens Axboe
A bit too sloppy on the handling of allocation failures at initialization time. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04blk-mq: style fixupsJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04multiqueue: change max queue depth from 256 to 2048Matias Bjørling
To prevent extensive use of wait-queue for benchmarking the null blk driver, we raise the value to 2048 for allowing at least 80 processes each with a queue depth of 32 to summit IOs. As request structures are allocated up front, we additionally do a graceful backscaling for queue depth if memory is not possible because of fragmentation. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04multiqueue: explicit define alloc/free for device driversMatias Bjørling
In the case where a device driver wish to implement its own allocation of hctx, it may overwrite the alloc_hctx and free_hctx operation. Device drivers which do not implement its own, should explicit (instead of implicit) use the blk_mq_[alloc/free]_single_queue mappings. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04null: allow utilization of per-node allocation of hctxsMatias Bjørling
In a multisocket system, it is beneficial to have one instance of the hardware context queue on each socket to minimize the amount of remote memory accesses required to serve an IO. This is similar to a block device that has multiple hardware queues, and wish to map on for each socket in the system. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04multiqueue: allow hctxs to be allocated by the block moduleMatias Bjørling
If a block device has multiple hardware queues, it may like to control where the hardware context queues are placed in memory to optimise contention points. For example: A NUMA system with four sockets and a block device that supports four hardware queues, may place each hardware queue context on each socket for better utilization of memory bandwidth. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04null: add queue depth configuration parameterMatias Bjørling
To allow the user to set the queue depth for each hardware queue, we expose a queue depth parameter for the user to pass at module init. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04blk-mq: style and moving of finish_wait()Jens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04virtio_blk: initial blk-mq supportJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04null: add test driverJens Axboe
The null driver just completes any request without transferring data. Useful for testing effectiveness of the IO stack. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04blk-mq: update TODOJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04blk-mq: don't attempt to create block trace points twiceJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04Export __smp_call_function_single() for modulesJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04blk-mq: add some commentsJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04Add hardware queue index for hctx in ctxJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04multiqueue: create check for hardware queues having no software queues attached.Matias Bjørling
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04multiqueue: mq_ops->map_queue can depend on the the index in ctx. We ↵Matias Bjørling
therefore initialize this structure before first calling hctx. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04multiqueue: trivial - fix up order of request queue initialization of ctx ↵Matias Bjørling
and hctx. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04multiqueue: initialize number of ctxs and hw_ctx before hctx initialization.Matias Bjørling
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04multiqueue: change ctx variable to per-cpu variable when passing ctx to driver.Matias Bjørling
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04multiqueue: add blktrace tracepointsJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04multiqueue: update MQ-TODOJens Axboe
- We can now limit queue depth successfully through our tag map. - We pass in REQ_END on the last request in a batch, this provides the needed information to the driver. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04multiqueue: pass in hctx to submit and end_io interfacesMatias Bjørling
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04multiqueue: improve timeout handlingJens Axboe
Should be closer to working now... Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04multiqueue: use tagging for depth and rq managementJens Axboe
Setup a tag map for the hardware queues. This provides us with the following: - Now we can limit depth as indicated by the driver - We can convert timeouts to just scan the array - Drivers get a tag value for free (most will need it) Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04multiqueue: stop merge checking if we hit a potential merge and failedJens Axboe
If blk_try_merge() returns that a merge would be possible and we failed merging, we have most likely hit segment (or other) restrictions. In that case, stop checking further, likelihood of a merge is then pretty much 0. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04multiqueue: shut up gccJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04multiqueue: updatesJens Axboe
- Make plugging work, also for merging. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04multiqueue: support alloc and insert of request on mqJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04multiqueue: ensure that blk_mq_init() is calledJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2012-10-04multiqueue: make IPI redirect be sysfs tweakableJens Axboe
Signed-off-by: Jens Axboe <axboe@kernel.dk>