summaryrefslogtreecommitdiff
path: root/drivers/lightnvm
AgeCommit message (Collapse)AuthorFilesLines
2016-01-12lightnvm: introduce factory resetMatias Bjørling2-0/+213
Now that a device can be managed using the system blocks, a method to reset the device is necessary as well. This patch introduces logic to reset the device easily to factory state and exposes it through an ioctl. The ioctl takes the following flags: NVM_FACTORY_ERASE_ONLY_USER By default all blocks, except host-reserved blocks are erased upon factory reset. Instead of this, only erase host-reserved blocks. NVM_FACTORY_RESET_HOST_BLKS Mark host-reserved blocks to be erased and set their type to free. NVM_FACTORY_RESET_GRWN_BBLKS Mark "grown bad blocks" to be erased and set their type to free. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12lightnvm: use system block for mm initializationMatias Bjørling1-2/+23
Use system block information to register the appropriate media manager. This enables the LightNVM subsystem to instantiate a media manager selected by the user, instead of relying on automatic detection by each media manager loaded in the kernel. A device must now be initialized before it can proceed to initialize its media manager. Upon initialization, the configured media manager is automatically initialized as well. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12lightnvm: introduce ioctl to initialize deviceMatias Bjørling1-0/+50
Based on the previous patch, we now introduce an ioctl to initialize the device using nvm_init_sysblock and create the necessary system blocks. The user may specify the media manager that they wish to instantiate on top. Default from user-space will be "gennvm". Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12lightnvm: core on-disk initializationMatias Bjørling3-1/+564
An Open-Channel SSD shall be initialized before use. To initialize, we define an on-disk format, that keeps a small set of metadata to bring up the media manager on top of the device. The initial step is introduced to allow a user to format the disks for a given media manager. During format, a system block is stored on one to three separate luns on the device. Each lun has the system block duplicated. During initialization, the system block can be retrieved and the appropriate media manager can initialized. The on-disk format currently covers (struct nvm_system_block): - Magic value "NVMS". - Monotonic increasing sequence number. - The physical block erase count. - Version of the system block format. - Media manager type. - Media manager superblock physical address. The interface provides three functions to manage the system block: int nvm_init_sysblock(struct nvm_dev *, struct nvm_sb_info *) int nvm_get_sysblock(struct nvm *dev, struct nvm_sb_info *) int nvm_update_sysblock(struct nvm *dev, struct nvm_sb_info *) Each implement a part of the logic to manage the system block. The initialization creates the first system blocks and mark them on the device. Get retrieves the latest system block by scanning all pages in the associated system blocks. The update sysblock writes new metadata and allocates new block if necessary. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12lightnvm: introduce mlc lower page table mappingsMatias Bjørling1-1/+60
NAND MLC memories have both lower and upper pages. When programming, both of these must be written, before data can be read. However, these lower and upper pages might not placed at even and odd flash pages, but can be skipped. Therefore each flash memory has its lower pages defined, which can then be used when programming and to know when padding are necessary. This patch implements the lower page definition in the specification, and exposes it through a simple lookup table at dev->lptbl. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12lightnvm: add mccap supportMatias Bjørling1-0/+1
Some flash media has extended capabilities, such as programming SLC pages on MLC/TLC flash, erase/program suspend, scramble and encryption. MCCAP is introduced to detect support for these capabilities in the command set. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12lightnvm: manage open and closed blocks separatelyJavier González4-43/+120
LightNVM targets need to know the state of the flash block when doing flash optimizations. An example is implementing a write buffer to respect the flash page size. Currently, block state is not accounted for; the media manager only differentiates among free, bad and in-use blocks. This patch adds the logic in the generic media manager to enable targets manage blocks into open and close separately, and it implements such management in rrpc. It also adds a set of flags to describe the state of the block (open, closed, free, bad). In order to avoid taking two locks (nvm_lun and rrpc_lun) consecutively, we introduce lockless get_/put_block primitives so that the open and close list locks and future common logic is handled within the nvm_lun lock. Signed-off-by: Javier González <javier@cnexlabs.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12lightnvm: reference rrpc lun in rrpc blockJavier González2-0/+2
Currently, a rrpc block only points to its nvm_lun. If a user wants to find the associated rrpc lun, it will have to calculate the index and look it up manually. By referencing the rrpc lun directly, this step can be omitted, at the cost of a larger memory footprint. This is important for upcoming patches that implement write buffering in rrpc. Signed-off-by: Javier González <javier@cnexlabs.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12lightnvm: introduce nvm_submit_ppaMatias Bjørling1-1/+44
Internal logic for both core and media managers, does not have a backing bio for issuing I/Os. Introduce nvm_submit_ppa to allow raw I/Os to be submitted to the underlying device driver. The function request the device, ppa, data buffer and its length and will submit the I/O synchronously to the device. The return value may therefore be used to detect any errors regarding the issued I/O. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12lightnvm: move rq->error to nvm_rq->errorMatias Bjørling3-5/+6
Instead of passing request error into the LightNVM modules, incorporate it into the nvm_rq. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12lightnvm: support multiple ppas in nvm_erase_ppaMatias Bjørling2-3/+3
Sometimes a user want to erase multiple PPAs at the same time. Extend nvm_erase_ppa to take multiple ppas and number of ppas to be erased. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12lightnvm: move the pages per block check out of the loopWenwei Tao1-6/+5
There is no need to check whether dev's pages per block is beyond rrpc support every time we init a lun, we only need to check it once before enter the lun init loop. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12lightnvm: sectors first in ppa listMatias Bjørling1-3/+3
The Westlake controller requires that the PPA list has sectors defined sequentially. Currently, the PPA list is created with planes first, then sectors. Change this to sectors first, then planes. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12lightnvm: fix locking and mempool in rrpc_lun_gcWenwei Tao1-6/+6
This patch fix two issues in rrpc_lun_gc 1. prio_list is protected by rrpc_lun's lock not nvm_lun's, so acquire rlun's lock instead of lun's before operate on the list. 2. we delete block from prio_list before allocating gcb, but gcb allocation may fail, we end without putting it back to the list, this makes the block won't get reclaimed in the future. To solve this issue, delete block after gcb allocation. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12lightnvm: put block back to gc list on its reclaim failWenwei Tao1-4/+14
We delete a block from the gc list before reclaim it, so put it back to the list on its reclaim fail, otherwise this block will not get reclaimed and be programmable in the future. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12lightnvm: check bi_error in gcWenwei Tao1-0/+6
We should check last io completion status before starting another one. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12lightnvm: refactor end_io functions for syncMatias Bjørling3-24/+32
To implement sync I/O support within the LightNVM core, the end_io functions are refactored to take an end_io function pointer instead of testing for initialized media manager, followed by calling its end_io function. Sync I/O can then be implemented using a callback that signal I/O completion. This is similar to the logic found in blk_to_execute_io(). By implementing it this way, the underlying device I/Os submission logic is abstracted away from core, targets, and media managers. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12lightnvm: refactor rqd ppa list into set/freeMatias Bjørling1-21/+50
A device may be driven in single, double or quad plane mode. In that case, the rqd must have either one, two, or four PPAs set for a single PPA sent to the device. Refactor this logic into their own functions to be shared by program/erase/read in the core. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12lightnvm: move ppa erase logic to coreMatias Bjørling2-64/+71
A device may function in single, dual or quad plane mode. The gennvm media manager manages this with explicit helpers. They convert a single ppa to 1, 2 or 4 separate ppas in a ppa list. To aid implementation of recovery and system blocks, this functionality can be moved directly into the core. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12lightnvm: unlock rq and free ppa_list on submission failWenwei Tao1-0/+6
When rrpc_write_ppalist_rq and rrpc_read_ppalist_rq succeed, we setup rq correctly, but nvm_submit_io may afterward fail since it cannot allocate request or nvme_nvm_command, we return error but forget to cleanup the previous work. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12lightnvm: add check after mempool allocationJavier Gonzalez1-0/+2
The mempool allocation might fail. Make sure to return error when it does, instead of causing a kernel panic. Signed-off-by: Javier Gonzalez <javier@cnexlabs.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12lightnvm: fix incorrect nr_free_blocks statChao Yu1-0/+1
When initing bad block list in gennvm_block_bb, once we move bad block from free_list to bb_list, we should maintain both stat info nr_free_blocks and nr_bad_blocks. So this patch fixes to add missing operation related to nr_free_blocks. Signed-off-by: Chao Yu <chao2.yu@samsung.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2016-01-12lightnvm: fix bio submission issueWenwei Tao2-2/+4
Put bio when submission fails, since we get it before submission. And return error when backend device driver doesn't provide a submit_io method, thus we can end IO properly. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-12-29lightnvm: wrong offset in bad blk lun calculationMatias Bjørling1-1/+1
dev->nr_luns reports the total number of luns available in a device while dev->luns_per_chnl is the number of luns per channel. When multiple channels are available, the offset is calculated from a channel and lun id into a linear array. As it multiplies with the total number of luns, we go out of bound when channel id > 0 and causes the kernel to panic when we read a protected kernel memory area. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-12-07lightnvm: do not compile in debugging by defaultMatias Bjørling1-0/+1
The LightNVM module exposes a debug interface when CONFIG_NVM_DEBUG is set. This interfaces takes a string to configure media managers and targets. Make sure this interface is only exposed when chosen deliberately. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-12-07lightnvm: prevent gennvm module unload on useMatias Bjørling1-0/+5
After the gennvm module has been initialized. It might be attached to one or several devices. In that case, the module is in use. Make sure that it can not be unloaded. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-12-07lightnvm: fix media mgr registrationMatias Bjørling1-39/+39
This patch fixes two issues during media manager registration. 1. The ppa pool can be used at media manager registration. Allocate the ppa pool before that. 2. If a media manager can't be found, this should not lead to the device being unallocated. A media manager can be registered later, that can manage the device. Only warn if a media manager fails initialization. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-12-07lightnvm: replace req queue with nvmdev for lldMatias Bjørling3-9/+8
In the case where a request queue is passed to the low lever lightnvm device drive integration, the device driver might pass its admin commands through another queue. Instead pass nvm_dev, and let the low level drive the appropriate queue. Reported-by: Christoph Hellwig <hch@infradead.org> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-12-07lightnvm: refactor spin_unlock in gennvm_get_blkWenwei Tao1-5/+2
The spin_unlock is duplicated multiple times. Jump to a single unlock to improve the code flow. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-12-07lightnvm: put blks when luns configure failedWenwei Tao1-2/+19
Put the allocated blocks back to the free list when the luns configure failed, to make these blocks useable to others. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-12-07lightnvm: use flags in rrpc_get_blkWenwei Tao1-1/+1
rrpc_get_blk use constant 0 as the input parameter of nvm_get_blk, this may result in getting gc block failed unexpectedly. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-30lightnvm: missing nvm_lock acquireWenwei Tao1-33/+42
To avoid race conditions, traverse dev, media manager, and target lists and also register, unregister entries to/from them, should be always under the nvm_lock control. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-30lightnvm: unconverted ppa returned in get_bb_tblMatias Bjørling1-2/+1
The get_bb_tbl function takes ppa as a generic address, which is converted to the ppa device address within the device driver. When the update_bbtbl callback is called from get_bb_tbl, the device specific ppa is used, instead of the generic ppa. Make sure to pass the generic ppa. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-30lightnvm: do device max sectors boundary check firstWenwei Tao1-4/+6
do device max_phys_sect boundary check first, otherwise we will allocate dma_pools for devices whose max sectors are beyond lightnvm support and register them. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-30lightnvm: fix ioctl memory leaksSudip Mukherjee1-2/+7
If copy_to_user() fails we returned error but we missed releasing devices. Signed-off-by: Sudip Mukherjee <sudip@vectorindia.org> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-30lightnvm: free memory when gennvm register failsWenwei Tao1-5/+10
free allocated nvm block and gennvm lun structures when gennvm register fails, otherwise it will cause memory leak. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-20lightnvm: add free and bad lun info to show lunsJavier Gonzalez2-6/+15
Add free block, used block, and bad block information to the show debug interface. This information is used to debug how targets track blocks. Also, change debug function name to make it more generic. Signed-off-by: Javier Gonzalez <javier@cnexlabs.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-20lightnvm: keep track of block countsJavier Gonzalez1-1/+13
Maintain number of in use blocks, free blocks, and bad blocks in a per lun basis. This allows the upper layers to get information about the state of each lun. Also, account for blocks reserved to the device on the free block count. nr_free_blocks matches now the actual number of blocks on the free list when the device is booted. Signed-off-by: Javier Gonzalez <javier@cnexlabs.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-20lightnvm: missing free on init errorMatias Bjørling1-2/+4
If either max_phys_sect is out of bound, the nvm_dev structure is not freed. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-20lightnvm: wrong return value and redundant freeWenwei Tao1-4/+2
The return value should be non-zero under error conditions. Remove nvme_free(dev) to avoid free dev more than once. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-17lightnvm: cleanup queue before target removalJavier González1-2/+2
This prevents outstanding IOs to be sent for completion to target after the target has been removed. The flow is now: stop new IOs > cleanup queue > remove target. Signed-off-by: Javier Gonzalez <javier@cnexlabs.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-17lightnvm: remove linear and device addr modesMatias Bjørling3-9/+38
The linear and device specific address modes can be replaced with a simple offset and bit length conversion that is generic across all devices. This both simplifies the specification and removes the special case for qemu nvme, that previously relied on the linear address mapping. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-17lightnvm: prevent double free on init errorMatias Bjørling1-9/+3
Both the nvm_register and nvm_init does a kfree(dev) on error. Make sure to only free it once. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-17lightnvm: prematurely activate nvm_devMatias Bjørling1-4/+4
We register with nvm_devices when there registration can still fail. Move the final registration at the end of the nvm_register function to make sure we are fully registered when added to the nvm_devices list. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-17lightnvm: check for NAND flash and its typeMatias Bjørling1-0/+10
Only NAND flash with SLC and MLC is supported. Make sure to not try to initialize TLC memory or other non-volatile memory types. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-17lightnvm: update bad block table formatMatias Bjørling2-10/+24
The specification was changed to reflect a multi-value bad block table. Instead of bit-based bad block table, the bad block table now allows eight bad block categories. Currently four are defined: * Factory bad blocks * Grown bad blocks * Device-side reserved blocks * Host-side reserved blocks The factory and grown bad blocks are the regular bad blocks. The reserved blocks are either for internal use or external use. In particular, the device-side reserved blocks allows the host to bootstrap from a limited number of flash blocks. Reducing the flash blocks to scan upon super block initialization. Support for both get bad block table and set bad block table is added. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-11-07block: change ->make_request_fn() and users to return a queue cookieJens Axboe1-4/+5
No functional changes in this patch, but it prepares us for returning a more useful cookie related to the IO that was queued up. Signed-off-by: Jens Axboe <axboe@fb.com> Acked-by: Christoph Hellwig <hch@lst.de> Acked-by: Keith Busch <keith.busch@intel.com>
2015-11-03lightnvm: refactor phys addrs type to u64Matias Bjørling2-12/+11
For cases where CONFIG_LBDAF is not set. The struct ppa_addr exceeds its type on 32 bit architectures. ppa_addr requires a 64bit integer to hold the generic ppa format. We therefore refactor it to u64 and replaces the sector_t usages with u64 for physical addresses. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29rrpc: Round-robin sector target with cost-based gcMatias Bjørling4-0/+1571
This target allows an Open-Channel SSD to be exposed asas a block device. It implements a round-robin approach for sector allocation, together with a greedy cost-based garbage collector. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
2015-10-29gennvm: Generic NVM managerMatias Bjørling4-0/+539
The implementation for Open-Channel SSDs is divided into media management and targets. This patch implements a generic media manager for open-channel SSDs. After a media manager has been initialized, single or multiple targets can be instantiated with the media managed as the backend. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
generated by cgit v1.2.3 (git 2.39.0) at 2024-12-27 23:31:02 +0300