From 311aab73d273eb22be976055f6cab224f7279d5e Mon Sep 17 00:00:00 2001 From: Colin Cross Date: Mon, 8 Aug 2011 23:39:36 +0200 Subject: PM / Runtime: Add might_sleep() to runtime PM functions Some of the entry points to pm runtime are not safe to call in atomic context unless pm_runtime_irq_safe() has been called. Inspecting the code, it is not immediately obvious that the functions sleep at all, as they run inside a spin_lock_irqsave, but under some conditions they can drop the lock and turn on irqs. If a driver incorrectly calls the pm_runtime apis, it can cause sleeping and irq processing when it expects to stay in atomic context. Add might_sleep_if to the majority of the __pm_runtime_* entry points to enforce correct usage. Add pm_runtime_put_sync_autosuspend to the list of functions that can be called in atomic context. Signed-off-by: Colin Cross Reviewed-by: Kevin Hilman Signed-off-by: Rafael J. Wysocki --- Documentation/power/runtime_pm.txt | 1 + 1 file changed, 1 insertion(+) (limited to 'Documentation/power') diff --git a/Documentation/power/runtime_pm.txt b/Documentation/power/runtime_pm.txt index 4ce5450ab6e8..62f37bc3866f 100644 --- a/Documentation/power/runtime_pm.txt +++ b/Documentation/power/runtime_pm.txt @@ -484,6 +484,7 @@ pm_runtime_resume() pm_runtime_get_sync() pm_runtime_put_sync() pm_runtime_put_sync_suspend() +pm_runtime_put_sync_autosuspend() 5. Runtime PM Initialization, Device Probing and Removal -- cgit v1.2.3 From 2e6ba515f50ef7ddf35b2703d014d3216c9b8b24 Mon Sep 17 00:00:00 2001 From: Ming Lei Date: Wed, 21 Sep 2011 22:31:33 +0200 Subject: PM / Runtime: pm_runtime_idle() can be called in atomic context Add to pm_runtime_idle the list of functions that can be called in atomic context if pm_runtime_irq_safe() has been called for the device. Signed-off-by: Ming Lei Signed-off-by: Rafael J. Wysocki --- Documentation/power/runtime_pm.txt | 1 + 1 file changed, 1 insertion(+) (limited to 'Documentation/power') diff --git a/Documentation/power/runtime_pm.txt b/Documentation/power/runtime_pm.txt index 62f37bc3866f..1750740b17e8 100644 --- a/Documentation/power/runtime_pm.txt +++ b/Documentation/power/runtime_pm.txt @@ -478,6 +478,7 @@ pm_runtime_autosuspend_expiration() If pm_runtime_irq_safe() has been called for a device then the following helper functions may also be used in interrupt context: +pm_runtime_idle() pm_runtime_suspend() pm_runtime_autosuspend() pm_runtime_resume() -- cgit v1.2.3 From e3cba3243eb853a052613c804dea033bc4c9cf2d Mon Sep 17 00:00:00 2001 From: Jean Pihet Date: Tue, 4 Oct 2011 21:54:45 +0200 Subject: PM / QoS: Update Documentation for the pm_qos and dev_pm_qos frameworks Update the documentation for the recently updated pm_qos API, kernel and user space. Add documentation for the per-device PM QoS (dev_pm_qos) framework API. Signed-off-by: Jean Pihet Signed-off-by: Rafael J. Wysocki --- Documentation/power/pm_qos_interface.txt | 92 ++++++++++++++++++++++++++++++-- 1 file changed, 87 insertions(+), 5 deletions(-) (limited to 'Documentation/power') diff --git a/Documentation/power/pm_qos_interface.txt b/Documentation/power/pm_qos_interface.txt index bfed898a03fc..17e130a80347 100644 --- a/Documentation/power/pm_qos_interface.txt +++ b/Documentation/power/pm_qos_interface.txt @@ -4,14 +4,19 @@ This interface provides a kernel and user mode interface for registering performance expectations by drivers, subsystems and user space applications on one of the parameters. -Currently we have {cpu_dma_latency, network_latency, network_throughput} as the -initial set of pm_qos parameters. +Two different PM QoS frameworks are available: +1. PM QoS classes for cpu_dma_latency, network_latency, network_throughput. +2. the per-device PM QoS framework provides the API to manage the per-device latency +constraints. Each parameters have defined units: * latency: usec * timeout: usec * throughput: kbs (kilo bit / sec) + +1. PM QoS framework + The infrastructure exposes multiple misc device nodes one per implemented parameter. The set of parameters implement is defined by pm_qos_power_init() and pm_qos_params.h. This is done because having the available parameters @@ -23,14 +28,18 @@ an aggregated target value. The aggregated target value is updated with changes to the request list or elements of the list. Typically the aggregated target value is simply the max or min of the request values held in the parameter list elements. +Note: the aggregated target value is implemented as an atomic variable so that +reading the aggregated value does not require any locking mechanism. + From kernel mode the use of this interface is simple: -handle = pm_qos_add_request(param_class, target_value): -Will insert an element into the list for that identified PM_QOS class with the +void pm_qos_add_request(handle, param_class, target_value): +Will insert an element into the list for that identified PM QoS class with the target value. Upon change to this list the new target is recomputed and any registered notifiers are called only if the target value is now different. -Clients of pm_qos need to save the returned handle. +Clients of pm_qos need to save the returned handle for future use in other +pm_qos API functions. void pm_qos_update_request(handle, new_target_value): Will update the list element pointed to by the handle with the new target value @@ -42,6 +51,20 @@ Will remove the element. After removal it will update the aggregate target and call the notification tree if the target was changed as a result of removing the request. +int pm_qos_request(param_class): +Returns the aggregated value for a given PM QoS class. + +int pm_qos_request_active(handle): +Returns if the request is still active, i.e. it has not been removed from a +PM QoS class constraints list. + +int pm_qos_add_notifier(param_class, notifier): +Adds a notification callback function to the PM QoS class. The callback is +called when the aggregated value for the PM QoS class is changed. + +int pm_qos_remove_notifier(int param_class, notifier): +Removes the notification callback function for the PM QoS class. + From user mode: Only processes can register a pm_qos request. To provide for automatic @@ -63,4 +86,63 @@ To remove the user mode request for a target value simply close the device node. +2. PM QoS per-device latency framework + +For each device a list of performance requests is maintained along with +an aggregated target value. The aggregated target value is updated with +changes to the request list or elements of the list. Typically the +aggregated target value is simply the max or min of the request values held +in the parameter list elements. +Note: the aggregated target value is implemented as an atomic variable so that +reading the aggregated value does not require any locking mechanism. + + +From kernel mode the use of this interface is the following: + +int dev_pm_qos_add_request(device, handle, value): +Will insert an element into the list for that identified device with the +target value. Upon change to this list the new target is recomputed and any +registered notifiers are called only if the target value is now different. +Clients of dev_pm_qos need to save the handle for future use in other +dev_pm_qos API functions. + +int dev_pm_qos_update_request(handle, new_value): +Will update the list element pointed to by the handle with the new target value +and recompute the new aggregated target, calling the notification trees if the +target is changed. + +int dev_pm_qos_remove_request(handle): +Will remove the element. After removal it will update the aggregate target and +call the notification trees if the target was changed as a result of removing +the request. + +s32 dev_pm_qos_read_value(device): +Returns the aggregated value for a given device's constraints list. + + +Notification mechanisms: +The per-device PM QoS framework has 2 different and distinct notification trees: +a per-device notification tree and a global notification tree. + +int dev_pm_qos_add_notifier(device, notifier): +Adds a notification callback function for the device. +The callback is called when the aggregated value of the device constraints list +is changed. + +int dev_pm_qos_remove_notifier(device, notifier): +Removes the notification callback function for the device. + +int dev_pm_qos_add_global_notifier(notifier): +Adds a notification callback function in the global notification tree of the +framework. +The callback is called when the aggregated value for any device is changed. + +int dev_pm_qos_remove_global_notifier(notifier): +Removes the notification callback function from the global notification tree +of the framework. + + +From user mode: +No API for user space access to the per-device latency constraints is provided +yet - still under discussion. -- cgit v1.2.3 From 2fb242adcaab5defa2f208775ac4f181ac998fdd Mon Sep 17 00:00:00 2001 From: Ming Lei Date: Sun, 9 Oct 2011 11:40:25 +0800 Subject: PM / Runtime: Update document about callbacks Support for device power domains has been introduced in commit 9659cc0678b954f187290c6e8b247a673c5d37e1 (PM: Make system-wide PM and runtime PM treat subsystems consistently), also power domain callbacks will take precedence over subsystem ones from commit 4d27e9dcff00a6425d779b065ec8892e4f391661(PM: Make power domain callbacks take precedence over subsystem ones). So update part of "Device Runtime PM Callbacks" in Documentation/power/runtime_pm.txt. Signed-off-by: Ming Lei Signed-off-by: Rafael J. Wysocki --- Documentation/power/runtime_pm.txt | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) (limited to 'Documentation/power') diff --git a/Documentation/power/runtime_pm.txt b/Documentation/power/runtime_pm.txt index 1750740b17e8..f670836f8e5f 100644 --- a/Documentation/power/runtime_pm.txt +++ b/Documentation/power/runtime_pm.txt @@ -43,13 +43,18 @@ struct dev_pm_ops { ... }; -The ->runtime_suspend(), ->runtime_resume() and ->runtime_idle() callbacks are -executed by the PM core for either the device type, or the class (if the device -type's struct dev_pm_ops object does not exist), or the bus type (if the -device type's and class' struct dev_pm_ops objects do not exist) of the given -device (this allows device types to override callbacks provided by bus types or -classes if necessary). The bus type, device type and class callbacks are -referred to as subsystem-level callbacks in what follows. +The ->runtime_suspend(), ->runtime_resume() and ->runtime_idle() callbacks +are executed by the PM core for either the power domain, or the device type +(if the device power domain's struct dev_pm_ops does not exist), or the class +(if the device power domain's and type's struct dev_pm_ops object does not +exist), or the bus type (if the device power domain's, type's and class' +struct dev_pm_ops objects do not exist) of the given device, so the priority +order of callbacks from high to low is that power domain callbacks, device +type callbacks, class callbacks and bus type callbacks, and the high priority +one will take precedence over low priority one. The bus type, device type and +class callbacks are referred to as subsystem-level callbacks in what follows, +and generally speaking, the power domain callbacks are used for representing +power domains within a SoC. By default, the callbacks are always invoked in process context with interrupts enabled. However, subsystems can use the pm_runtime_irq_safe() helper function -- cgit v1.2.3 From 2a77c46de1e3dace73745015635ebbc648eca69c Mon Sep 17 00:00:00 2001 From: ShuoX Liu Date: Wed, 10 Aug 2011 23:01:26 +0200 Subject: PM / Suspend: Add statistics debugfs file for suspend to RAM Record S3 failure time about each reason and the latest two failed devices' names in S3 progress. We can check it through 'suspend_stats' entry in debugfs. The motivation of the patch: We are enabling power features on Medfield. Comparing with PC/notebook, a mobile enters/exits suspend-2-ram (we call it s3 on Medfield) far more frequently. If it can't enter suspend-2-ram in time, the power might be used up soon. We often find sometimes, a device suspend fails. Then, system retries s3 over and over again. As display is off, testers and developers don't know what happens. Some testers and developers complain they don't know if system tries suspend-2-ram, and what device fails to suspend. They need such info for a quick check. The patch adds suspend_stats under debugfs for users to check suspend to RAM statistics quickly. If not using this patch, we have other methods to get info about what device fails. One is to turn on CONFIG_PM_DEBUG, but users would get too much info and testers need recompile the system. In addition, dynamic debug is another good tool to dump debug info. But it still doesn't match our utilization scenario closely. 1) user need write a user space parser to process the syslog output; 2) Our testing scenario is we leave the mobile for at least hours. Then, check its status. No serial console available during the testing. One is because console would be suspended, and the other is serial console connecting with spi or HSU devices would consume power. These devices are powered off at suspend-2-ram. Signed-off-by: ShuoX Liu Signed-off-by: Rafael J. Wysocki --- Documentation/power/basic-pm-debugging.txt | 24 +++++++ drivers/base/power/main.c | 31 +++++++-- include/linux/suspend.h | 52 +++++++++++++++ kernel/power/main.c | 102 +++++++++++++++++++++++++++++ kernel/power/suspend.c | 17 ++++- 5 files changed, 218 insertions(+), 8 deletions(-) (limited to 'Documentation/power') diff --git a/Documentation/power/basic-pm-debugging.txt b/Documentation/power/basic-pm-debugging.txt index ddd78172ef73..62eca080a71b 100644 --- a/Documentation/power/basic-pm-debugging.txt +++ b/Documentation/power/basic-pm-debugging.txt @@ -201,3 +201,27 @@ case, you may be able to search for failing drivers by following the procedure analogous to the one described in section 1. If you find some failing drivers, you will have to unload them every time before an STR transition (ie. before you run s2ram), and please report the problems with them. + +There is a debugfs entry which shows the suspend to RAM statistics. Here is an +example of its output. + # mount -t debugfs none /sys/kernel/debug + # cat /sys/kernel/debug/suspend_stats + success: 20 + fail: 5 + failed_freeze: 0 + failed_prepare: 0 + failed_suspend: 5 + failed_suspend_noirq: 0 + failed_resume: 0 + failed_resume_noirq: 0 + failures: + last_failed_dev: alarm + adc + last_failed_errno: -16 + -16 + last_failed_step: suspend + suspend +Field success means the success number of suspend to RAM, and field fail means +the failure number. Others are the failure number of different steps of suspend +to RAM. suspend_stats just lists the last 2 failed devices, error number and +failed step of suspend. diff --git a/drivers/base/power/main.c b/drivers/base/power/main.c index c6291ab725a3..b1b58260b4ff 100644 --- a/drivers/base/power/main.c +++ b/drivers/base/power/main.c @@ -46,6 +46,7 @@ LIST_HEAD(dpm_prepared_list); LIST_HEAD(dpm_suspended_list); LIST_HEAD(dpm_noirq_list); +struct suspend_stats suspend_stats; static DEFINE_MUTEX(dpm_list_mtx); static pm_message_t pm_transition; @@ -467,8 +468,12 @@ void dpm_resume_noirq(pm_message_t state) mutex_unlock(&dpm_list_mtx); error = device_resume_noirq(dev, state); - if (error) + if (error) { + suspend_stats.failed_resume_noirq++; + dpm_save_failed_step(SUSPEND_RESUME_NOIRQ); + dpm_save_failed_dev(dev_name(dev)); pm_dev_err(dev, state, " early", error); + } mutex_lock(&dpm_list_mtx); put_device(dev); @@ -629,8 +634,12 @@ void dpm_resume(pm_message_t state) mutex_unlock(&dpm_list_mtx); error = device_resume(dev, state, false); - if (error) + if (error) { + suspend_stats.failed_resume++; + dpm_save_failed_step(SUSPEND_RESUME); + dpm_save_failed_dev(dev_name(dev)); pm_dev_err(dev, state, "", error); + } mutex_lock(&dpm_list_mtx); } @@ -805,6 +814,9 @@ int dpm_suspend_noirq(pm_message_t state) mutex_lock(&dpm_list_mtx); if (error) { pm_dev_err(dev, state, " late", error); + suspend_stats.failed_suspend_noirq++; + dpm_save_failed_step(SUSPEND_SUSPEND_NOIRQ); + dpm_save_failed_dev(dev_name(dev)); put_device(dev); break; } @@ -926,8 +938,10 @@ static void async_suspend(void *data, async_cookie_t cookie) int error; error = __device_suspend(dev, pm_transition, true); - if (error) + if (error) { + dpm_save_failed_dev(dev_name(dev)); pm_dev_err(dev, pm_transition, " async", error); + } put_device(dev); } @@ -970,6 +984,7 @@ int dpm_suspend(pm_message_t state) mutex_lock(&dpm_list_mtx); if (error) { pm_dev_err(dev, state, "", error); + dpm_save_failed_dev(dev_name(dev)); put_device(dev); break; } @@ -983,7 +998,10 @@ int dpm_suspend(pm_message_t state) async_synchronize_full(); if (!error) error = async_error; - if (!error) + if (error) { + suspend_stats.failed_suspend++; + dpm_save_failed_step(SUSPEND_SUSPEND); + } else dpm_show_time(starttime, state, NULL); return error; } @@ -1091,7 +1109,10 @@ int dpm_suspend_start(pm_message_t state) int error; error = dpm_prepare(state); - if (!error) + if (error) { + suspend_stats.failed_prepare++; + dpm_save_failed_step(SUSPEND_PREPARE); + } else error = dpm_suspend(state); return error; } diff --git a/include/linux/suspend.h b/include/linux/suspend.h index 6bbcef22e105..76f42e49b72d 100644 --- a/include/linux/suspend.h +++ b/include/linux/suspend.h @@ -34,6 +34,58 @@ typedef int __bitwise suspend_state_t; #define PM_SUSPEND_MEM ((__force suspend_state_t) 3) #define PM_SUSPEND_MAX ((__force suspend_state_t) 4) +enum suspend_stat_step { + SUSPEND_FREEZE = 1, + SUSPEND_PREPARE, + SUSPEND_SUSPEND, + SUSPEND_SUSPEND_NOIRQ, + SUSPEND_RESUME_NOIRQ, + SUSPEND_RESUME +}; + +struct suspend_stats { + int success; + int fail; + int failed_freeze; + int failed_prepare; + int failed_suspend; + int failed_suspend_noirq; + int failed_resume; + int failed_resume_noirq; +#define REC_FAILED_NUM 2 + int last_failed_dev; + char failed_devs[REC_FAILED_NUM][40]; + int last_failed_errno; + int errno[REC_FAILED_NUM]; + int last_failed_step; + enum suspend_stat_step failed_steps[REC_FAILED_NUM]; +}; + +extern struct suspend_stats suspend_stats; + +static inline void dpm_save_failed_dev(const char *name) +{ + strlcpy(suspend_stats.failed_devs[suspend_stats.last_failed_dev], + name, + sizeof(suspend_stats.failed_devs[0])); + suspend_stats.last_failed_dev++; + suspend_stats.last_failed_dev %= REC_FAILED_NUM; +} + +static inline void dpm_save_failed_errno(int err) +{ + suspend_stats.errno[suspend_stats.last_failed_errno] = err; + suspend_stats.last_failed_errno++; + suspend_stats.last_failed_errno %= REC_FAILED_NUM; +} + +static inline void dpm_save_failed_step(enum suspend_stat_step step) +{ + suspend_stats.failed_steps[suspend_stats.last_failed_step] = step; + suspend_stats.last_failed_step++; + suspend_stats.last_failed_step %= REC_FAILED_NUM; +} + /** * struct platform_suspend_ops - Callbacks for managing platform dependent * system sleep states. diff --git a/kernel/power/main.c b/kernel/power/main.c index 6c601f871964..2757acba8e8a 100644 --- a/kernel/power/main.c +++ b/kernel/power/main.c @@ -12,6 +12,8 @@ #include #include #include +#include +#include #include "power.h" @@ -133,6 +135,101 @@ power_attr(pm_test); #endif /* CONFIG_PM_SLEEP */ +#ifdef CONFIG_DEBUG_FS +static char *suspend_step_name(enum suspend_stat_step step) +{ + switch (step) { + case SUSPEND_FREEZE: + return "freeze"; + case SUSPEND_PREPARE: + return "prepare"; + case SUSPEND_SUSPEND: + return "suspend"; + case SUSPEND_SUSPEND_NOIRQ: + return "suspend_noirq"; + case SUSPEND_RESUME_NOIRQ: + return "resume_noirq"; + case SUSPEND_RESUME: + return "resume"; + default: + return ""; + } +} + +static int suspend_stats_show(struct seq_file *s, void *unused) +{ + int i, index, last_dev, last_errno, last_step; + + last_dev = suspend_stats.last_failed_dev + REC_FAILED_NUM - 1; + last_dev %= REC_FAILED_NUM; + last_errno = suspend_stats.last_failed_errno + REC_FAILED_NUM - 1; + last_errno %= REC_FAILED_NUM; + last_step = suspend_stats.last_failed_step + REC_FAILED_NUM - 1; + last_step %= REC_FAILED_NUM; + seq_printf(s, "%s: %d\n%s: %d\n%s: %d\n%s: %d\n" + "%s: %d\n%s: %d\n%s: %d\n%s: %d\n", + "success", suspend_stats.success, + "fail", suspend_stats.fail, + "failed_freeze", suspend_stats.failed_freeze, + "failed_prepare", suspend_stats.failed_prepare, + "failed_suspend", suspend_stats.failed_suspend, + "failed_suspend_noirq", + suspend_stats.failed_suspend_noirq, + "failed_resume", suspend_stats.failed_resume, + "failed_resume_noirq", + suspend_stats.failed_resume_noirq); + seq_printf(s, "failures:\n last_failed_dev:\t%-s\n", + suspend_stats.failed_devs[last_dev]); + for (i = 1; i < REC_FAILED_NUM; i++) { + index = last_dev + REC_FAILED_NUM - i; + index %= REC_FAILED_NUM; + seq_printf(s, "\t\t\t%-s\n", + suspend_stats.failed_devs[index]); + } + seq_printf(s, " last_failed_errno:\t%-d\n", + suspend_stats.errno[last_errno]); + for (i = 1; i < REC_FAILED_NUM; i++) { + index = last_errno + REC_FAILED_NUM - i; + index %= REC_FAILED_NUM; + seq_printf(s, "\t\t\t%-d\n", + suspend_stats.errno[index]); + } + seq_printf(s, " last_failed_step:\t%-s\n", + suspend_step_name( + suspend_stats.failed_steps[last_step])); + for (i = 1; i < REC_FAILED_NUM; i++) { + index = last_step + REC_FAILED_NUM - i; + index %= REC_FAILED_NUM; + seq_printf(s, "\t\t\t%-s\n", + suspend_step_name( + suspend_stats.failed_steps[index])); + } + + return 0; +} + +static int suspend_stats_open(struct inode *inode, struct file *file) +{ + return single_open(file, suspend_stats_show, NULL); +} + +static const struct file_operations suspend_stats_operations = { + .open = suspend_stats_open, + .read = seq_read, + .llseek = seq_lseek, + .release = single_release, +}; + +static int __init pm_debugfs_init(void) +{ + debugfs_create_file("suspend_stats", S_IFREG | S_IRUGO, + NULL, NULL, &suspend_stats_operations); + return 0; +} + +late_initcall(pm_debugfs_init); +#endif /* CONFIG_DEBUG_FS */ + struct kobject *power_kobj; /** @@ -194,6 +291,11 @@ static ssize_t state_store(struct kobject *kobj, struct kobj_attribute *attr, } if (state < PM_SUSPEND_MAX && *s) error = enter_state(state); + if (error) { + suspend_stats.fail++; + dpm_save_failed_errno(error); + } else + suspend_stats.success++; #endif Exit: diff --git a/kernel/power/suspend.c b/kernel/power/suspend.c index b6b71ad2208f..595a3dd56a8a 100644 --- a/kernel/power/suspend.c +++ b/kernel/power/suspend.c @@ -104,7 +104,10 @@ static int suspend_prepare(void) goto Finish; error = suspend_freeze_processes(); - if (!error) + if (error) { + suspend_stats.failed_freeze++; + dpm_save_failed_step(SUSPEND_FREEZE); + } else return 0; suspend_thaw_processes(); @@ -315,8 +318,16 @@ int enter_state(suspend_state_t state) */ int pm_suspend(suspend_state_t state) { - if (state > PM_SUSPEND_ON && state <= PM_SUSPEND_MAX) - return enter_state(state); + int ret; + if (state > PM_SUSPEND_ON && state <= PM_SUSPEND_MAX) { + ret = enter_state(state); + if (ret) { + suspend_stats.fail++; + dpm_save_failed_errno(ret); + } else + suspend_stats.success++; + return ret; + } return -EINVAL; } EXPORT_SYMBOL(pm_suspend); -- cgit v1.2.3 From 8f88893c05f2f677f18f2ce5591b4bed5d4a7535 Mon Sep 17 00:00:00 2001 From: Alan Stern Date: Mon, 26 Sep 2011 17:38:50 +0200 Subject: PM: Update the policy on default wakeup settings This patch (as1485) documents a change to the kernel's default wakeup policy. Devices that forward wakeup requests between buses should be enabled for wakeup by default. Signed-off-by: Alan Stern Signed-off-by: Rafael J. Wysocki --- Documentation/power/devices.txt | 4 +++- drivers/base/power/wakeup.c | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-) (limited to 'Documentation/power') diff --git a/Documentation/power/devices.txt b/Documentation/power/devices.txt index 3384d5996be2..29b7a9817f5a 100644 --- a/Documentation/power/devices.txt +++ b/Documentation/power/devices.txt @@ -152,7 +152,9 @@ try to use its wakeup mechanism. device_set_wakeup_enable() affects this flag; for the most part drivers should not change its value. The initial value of should_wakeup is supposed to be false for the majority of devices; the major exceptions are power buttons, keyboards, and Ethernet adapters whose WoL -(wake-on-LAN) feature has been set up with ethtool. +(wake-on-LAN) feature has been set up with ethtool. It should also default +to true for devices that don't generate wakeup requests on their own but merely +forward wakeup requests from one bus to another (like PCI bridges). Whether or not a device is capable of issuing wakeup events is a hardware matter, and the kernel is responsible for keeping track of it. By contrast, diff --git a/drivers/base/power/wakeup.c b/drivers/base/power/wakeup.c index 84f7c7d5a098..14ee07e9cc43 100644 --- a/drivers/base/power/wakeup.c +++ b/drivers/base/power/wakeup.c @@ -276,7 +276,9 @@ EXPORT_SYMBOL_GPL(device_set_wakeup_capable); * * By default, most devices should leave wakeup disabled. The exceptions are * devices that everyone expects to be wakeup sources: keyboards, power buttons, - * possibly network interfaces, etc. + * possibly network interfaces, etc. Also, devices that don't generate their + * own wakeup requests but merely forward requests from one bus to another + * (like PCI bridges) should have wakeup enabled by default. */ int device_init_wakeup(struct device *dev, bool enable) { -- cgit v1.2.3 From 2aede851ddf08666f68ffc17be446420e9d2a056 Mon Sep 17 00:00:00 2001 From: "Rafael J. Wysocki" Date: Mon, 26 Sep 2011 20:32:27 +0200 Subject: PM / Hibernate: Freeze kernel threads after preallocating memory There is a problem with the current ordering of hibernate code which leads to deadlocks in some filesystems' memory shrinkers. Namely, some filesystems use freezable kernel threads that are inactive when the hibernate memory preallocation is carried out. Those same filesystems use memory shrinkers that may be triggered by the hibernate memory preallocation. If those memory shrinkers wait for the frozen kernel threads, the hibernate process deadlocks (this happens with XFS, for one example). Apparently, it is not technically viable to redesign the filesystems in question to avoid the situation described above, so the only possible solution of this issue is to defer the freezing of kernel threads until the hibernate memory preallocation is done, which is implemented by this change. Unfortunately, this requires the memory preallocation to be done before the "prepare" stage of device freeze, so after this change the only way drivers can allocate additional memory for their freeze routines in a clean way is to use PM notifiers. Reported-by: Christoph Signed-off-by: Rafael J. Wysocki --- Documentation/power/devices.txt | 4 ---- include/linux/freezer.h | 4 +++- kernel/power/hibernate.c | 12 ++++++++---- kernel/power/power.h | 3 ++- kernel/power/process.c | 30 ++++++++++++++++++++---------- 5 files changed, 33 insertions(+), 20 deletions(-) (limited to 'Documentation/power') diff --git a/Documentation/power/devices.txt b/Documentation/power/devices.txt index 29b7a9817f5a..646a89e0c07d 100644 --- a/Documentation/power/devices.txt +++ b/Documentation/power/devices.txt @@ -281,10 +281,6 @@ When the system goes into the standby or memory sleep state, the phases are: time.) Unlike the other suspend-related phases, during the prepare phase the device tree is traversed top-down. - In addition to that, if device drivers need to allocate additional - memory to be able to hadle device suspend correctly, that should be - done in the prepare phase. - After the prepare callback method returns, no new children may be registered below the device. The method may also prepare the device or driver in some way for the upcoming system power transition (for diff --git a/include/linux/freezer.h b/include/linux/freezer.h index 1effc8b56b4e..aa56cf31f7ff 100644 --- a/include/linux/freezer.h +++ b/include/linux/freezer.h @@ -49,6 +49,7 @@ extern int thaw_process(struct task_struct *p); extern void refrigerator(void); extern int freeze_processes(void); +extern int freeze_kernel_threads(void); extern void thaw_processes(void); static inline int try_to_freeze(void) @@ -171,7 +172,8 @@ static inline void clear_freeze_flag(struct task_struct *p) {} static inline int thaw_process(struct task_struct *p) { return 1; } static inline void refrigerator(void) {} -static inline int freeze_processes(void) { BUG(); return 0; } +static inline int freeze_processes(void) { return -ENOSYS; } +static inline int freeze_kernel_threads(void) { return -ENOSYS; } static inline void thaw_processes(void) {} static inline int try_to_freeze(void) { return 0; } diff --git a/kernel/power/hibernate.c b/kernel/power/hibernate.c index 8f7b1db1ece1..3a20466015f8 100644 --- a/kernel/power/hibernate.c +++ b/kernel/power/hibernate.c @@ -334,12 +334,16 @@ int hibernation_snapshot(int platform_mode) if (error) goto Close; - error = dpm_prepare(PMSG_FREEZE); - if (error) - goto Complete_devices; - /* Preallocate image memory before shutting down devices. */ error = hibernate_preallocate_memory(); + if (error) + goto Close; + + error = freeze_kernel_threads(); + if (error) + goto Close; + + error = dpm_prepare(PMSG_FREEZE); if (error) goto Complete_devices; diff --git a/kernel/power/power.h b/kernel/power/power.h index 9a00a0a26280..e6206397ce67 100644 --- a/kernel/power/power.h +++ b/kernel/power/power.h @@ -228,7 +228,8 @@ extern int pm_test_level; #ifdef CONFIG_SUSPEND_FREEZER static inline int suspend_freeze_processes(void) { - return freeze_processes(); + int error = freeze_processes(); + return error ? : freeze_kernel_threads(); } static inline void suspend_thaw_processes(void) diff --git a/kernel/power/process.c b/kernel/power/process.c index 0cf3a27a6c9d..addbbe5531bc 100644 --- a/kernel/power/process.c +++ b/kernel/power/process.c @@ -135,7 +135,7 @@ static int try_to_freeze_tasks(bool sig_only) } /** - * freeze_processes - tell processes to enter the refrigerator + * freeze_processes - Signal user space processes to enter the refrigerator. */ int freeze_processes(void) { @@ -143,20 +143,30 @@ int freeze_processes(void) printk("Freezing user space processes ... "); error = try_to_freeze_tasks(true); - if (error) - goto Exit; - printk("done.\n"); + if (!error) { + printk("done."); + oom_killer_disable(); + } + printk("\n"); + BUG_ON(in_atomic()); + + return error; +} + +/** + * freeze_kernel_threads - Make freezable kernel threads go to the refrigerator. + */ +int freeze_kernel_threads(void) +{ + int error; printk("Freezing remaining freezable tasks ... "); error = try_to_freeze_tasks(false); - if (error) - goto Exit; - printk("done."); + if (!error) + printk("done."); - oom_killer_disable(); - Exit: - BUG_ON(in_atomic()); printk("\n"); + BUG_ON(in_atomic()); return error; } -- cgit v1.2.3 From 7fef9fc83fbd7293ea9fe665d14046422ebf4219 Mon Sep 17 00:00:00 2001 From: "Srivatsa S. Bhat" Date: Wed, 19 Oct 2011 23:59:05 +0200 Subject: PM / Documentation: Update docs about suspend and CPU hotplug Update the documentation about the interaction between the suspend (S3) call path and the CPU hotplug infrastructure. This patch focusses only on the activities of the freezer, cpu hotplug and the notifications involved. It outlines how regular CPU hotplug differs from the way it is invoked during suspend and also tries to explain the locking involved. In addition to that, it discusses the issue of microcode update during CPU hotplug operations. Signed-off-by: Srivatsa S. Bhat Signed-off-by: Rafael J. Wysocki --- Documentation/power/00-INDEX | 2 + Documentation/power/suspend-and-cpuhotplug.txt | 275 +++++++++++++++++++++++++ 2 files changed, 277 insertions(+) create mode 100644 Documentation/power/suspend-and-cpuhotplug.txt (limited to 'Documentation/power') diff --git a/Documentation/power/00-INDEX b/Documentation/power/00-INDEX index 45e9d4a91284..a4d682f54231 100644 --- a/Documentation/power/00-INDEX +++ b/Documentation/power/00-INDEX @@ -26,6 +26,8 @@ s2ram.txt - How to get suspend to ram working (and debug it when it isn't) states.txt - System power management states +suspend-and-cpuhotplug.txt + - Explains the interaction between Suspend-to-RAM (S3) and CPU hotplug swsusp-and-swap-files.txt - Using swap files with software suspend (to disk) swsusp-dmcrypt.txt diff --git a/Documentation/power/suspend-and-cpuhotplug.txt b/Documentation/power/suspend-and-cpuhotplug.txt new file mode 100644 index 000000000000..f28f9a6f0347 --- /dev/null +++ b/Documentation/power/suspend-and-cpuhotplug.txt @@ -0,0 +1,275 @@ +Interaction of Suspend code (S3) with the CPU hotplug infrastructure + + (C) 2011 Srivatsa S. Bhat + + +I. How does the regular CPU hotplug code differ from how the Suspend-to-RAM + infrastructure uses it internally? And where do they share common code? + +Well, a picture is worth a thousand words... So ASCII art follows :-) + +[This depicts the current design in the kernel, and focusses only on the +interactions involving the freezer and CPU hotplug and also tries to explain +the locking involved. It outlines the notifications involved as well. +But please note that here, only the call paths are illustrated, with the aim +of describing where they take different paths and where they share code. +What happens when regular CPU hotplug and Suspend-to-RAM race with each other +is not depicted here.] + +On a high level, the suspend-resume cycle goes like this: + +|Freeze| -> |Disable nonboot| -> |Do suspend| -> |Enable nonboot| -> |Thaw | +|tasks | | cpus | | | | cpus | |tasks| + + +More details follow: + + Suspend call path + ----------------- + + Write 'mem' to + /sys/power/state + syfs file + | + v + Acquire pm_mutex lock + | + v + Send PM_SUSPEND_PREPARE + notifications + | + v + Freeze tasks + | + | + v + disable_nonboot_cpus() + /* start */ + | + v + Acquire cpu_add_remove_lock + | + v + Iterate over CURRENTLY + online CPUs + | + | + | ---------- + v | L + ======> _cpu_down() | + | [This takes cpuhotplug.lock | + Common | before taking down the CPU | + code | and releases it when done] | O + | While it is at it, notifications | + | are sent when notable events occur, | + ======> by running all registered callbacks. | + | | O + | | + | | + v | + Note down these cpus in | P + frozen_cpus mask ---------- + | + v + Disable regular cpu hotplug + by setting cpu_hotplug_disabled=1 + | + v + Release cpu_add_remove_lock + | + v + /* disable_nonboot_cpus() complete */ + | + v + Do suspend + + + +Resuming back is likewise, with the counterparts being (in the order of +execution during resume): +* enable_nonboot_cpus() which involves: + | Acquire cpu_add_remove_lock + | Reset cpu_hotplug_disabled to 0, thereby enabling regular cpu hotplug + | Call _cpu_up() [for all those cpus in the frozen_cpus mask, in a loop] + | Release cpu_add_remove_lock + v + +* thaw tasks +* send PM_POST_SUSPEND notifications +* Release pm_mutex lock. + + +It is to be noted here that the pm_mutex lock is acquired at the very +beginning, when we are just starting out to suspend, and then released only +after the entire cycle is complete (i.e., suspend + resume). + + + + Regular CPU hotplug call path + ----------------------------- + + Write 0 (or 1) to + /sys/devices/system/cpu/cpu*/online + sysfs file + | + | + v + cpu_down() + | + v + Acquire cpu_add_remove_lock + | + v + If cpu_hotplug_disabled is 1 + return gracefully + | + | + v + ======> _cpu_down() + | [This takes cpuhotplug.lock + Common | before taking down the CPU + code | and releases it when done] + | While it is at it, notifications + | are sent when notable events occur, + ======> by running all registered callbacks. + | + | + v + Release cpu_add_remove_lock + [That's it!, for + regular CPU hotplug] + + + +So, as can be seen from the two diagrams (the parts marked as "Common code"), +regular CPU hotplug and the suspend code path converge at the _cpu_down() and +_cpu_up() functions. They differ in the arguments passed to these functions, +in that during regular CPU hotplug, 0 is passed for the 'tasks_frozen' +argument. But during suspend, since the tasks are already frozen by the time +the non-boot CPUs are offlined or onlined, the _cpu_*() functions are called +with the 'tasks_frozen' argument set to 1. +[See below for some known issues regarding this.] + + +Important files and functions/entry points: +------------------------------------------ + +kernel/power/process.c : freeze_processes(), thaw_processes() +kernel/power/suspend.c : suspend_prepare(), suspend_enter(), suspend_finish() +kernel/cpu.c: cpu_[up|down](), _cpu_[up|down](), [disable|enable]_nonboot_cpus() + + + +II. What are the issues involved in CPU hotplug? + ------------------------------------------- + +There are some interesting situations involving CPU hotplug and microcode +update on the CPUs, as discussed below: + +[Please bear in mind that the kernel requests the microcode images from +userspace, using the request_firmware() function defined in +drivers/base/firmware_class.c] + + +a. When all the CPUs are identical: + + This is the most common situation and it is quite straightforward: we want + to apply the same microcode revision to each of the CPUs. + To give an example of x86, the collect_cpu_info() function defined in + arch/x86/kernel/microcode_core.c helps in discovering the type of the CPU + and thereby in applying the correct microcode revision to it. + But note that the kernel does not maintain a common microcode image for the + all CPUs, in order to handle case 'b' described below. + + +b. When some of the CPUs are different than the rest: + + In this case since we probably need to apply different microcode revisions + to different CPUs, the kernel maintains a copy of the correct microcode + image for each CPU (after appropriate CPU type/model discovery using + functions such as collect_cpu_info()). + + +c. When a CPU is physically hot-unplugged and a new (and possibly different + type of) CPU is hot-plugged into the system: + + In the current design of the kernel, whenever a CPU is taken offline during + a regular CPU hotplug operation, upon receiving the CPU_DEAD notification + (which is sent by the CPU hotplug code), the microcode update driver's + callback for that event reacts by freeing the kernel's copy of the + microcode image for that CPU. + + Hence, when a new CPU is brought online, since the kernel finds that it + doesn't have the microcode image, it does the CPU type/model discovery + afresh and then requests the userspace for the appropriate microcode image + for that CPU, which is subsequently applied. + + For example, in x86, the mc_cpu_callback() function (which is the microcode + update driver's callback registered for CPU hotplug events) calls + microcode_update_cpu() which would call microcode_init_cpu() in this case, + instead of microcode_resume_cpu() when it finds that the kernel doesn't + have a valid microcode image. This ensures that the CPU type/model + discovery is performed and the right microcode is applied to the CPU after + getting it from userspace. + + +d. Handling microcode update during suspend/hibernate: + + Strictly speaking, during a CPU hotplug operation which does not involve + physically removing or inserting CPUs, the CPUs are not actually powered + off during a CPU offline. They are just put to the lowest C-states possible. + Hence, in such a case, it is not really necessary to re-apply microcode + when the CPUs are brought back online, since they wouldn't have lost the + image during the CPU offline operation. + + This is the usual scenario encountered during a resume after a suspend. + However, in the case of hibernation, since all the CPUs are completely + powered off, during restore it becomes necessary to apply the microcode + images to all the CPUs. + + [Note that we don't expect someone to physically pull out nodes and insert + nodes with a different type of CPUs in-between a suspend-resume or a + hibernate/restore cycle.] + + In the current design of the kernel however, during a CPU offline operation + as part of the suspend/hibernate cycle (the CPU_DEAD_FROZEN notification), + the existing copy of microcode image in the kernel is not freed up. + And during the CPU online operations (during resume/restore), since the + kernel finds that it already has copies of the microcode images for all the + CPUs, it just applies them to the CPUs, avoiding any re-discovery of CPU + type/model and the need for validating whether the microcode revisions are + right for the CPUs or not (due to the above assumption that physical CPU + hotplug will not be done in-between suspend/resume or hibernate/restore + cycles). + + +III. Are there any known problems when regular CPU hotplug and suspend race + with each other? + +Yes, they are listed below: + +1. When invoking regular CPU hotplug, the 'tasks_frozen' argument passed to + the _cpu_down() and _cpu_up() functions is *always* 0. + This might not reflect the true current state of the system, since the + tasks could have been frozen by an out-of-band event such as a suspend + operation in progress. Hence, it will lead to wrong notifications being + sent during the cpu online/offline events (eg, CPU_ONLINE notification + instead of CPU_ONLINE_FROZEN) which in turn will lead to execution of + inappropriate code by the callbacks registered for such CPU hotplug events. + +2. If a regular CPU hotplug stress test happens to race with the freezer due + to a suspend operation in progress at the same time, then we could hit the + situation described below: + + * A regular cpu online operation continues its journey from userspace + into the kernel, since the freezing has not yet begun. + * Then freezer gets to work and freezes userspace. + * If cpu online has not yet completed the microcode update stuff by now, + it will now start waiting on the frozen userspace in the + TASK_UNINTERRUPTIBLE state, in order to get the microcode image. + * Now the freezer continues and tries to freeze the remaining tasks. But + due to this wait mentioned above, the freezer won't be able to freeze + the cpu online hotplug task and hence freezing of tasks fails. + + As a result of this task freezing failure, the suspend operation gets + aborted. -- cgit v1.2.3