container_content retrieves volume information from disks in the
container. For unsupported volumes the function was not returning
mdinfo. When all volumes were unsupported the function was returning
NULL pointer to block actions on the volumes. Therefore, such volumes
were not activated in Incremental and Assembly. As side effect they
also could not be deleted using kill-subarray since "kill" function
requires to obtain a valid mdinfo from container_content.
This patch fixes the kill-subarray problem by allowing to obtain
mdinfo of all volumes types including unsupported and introducing new
array.status flags.
There are following changes:
1. Added MD_SB_BLOCK_VOLUME for blocking an array, other arrays in the
container can be activated.
2. Added MD_SB_BLOCK_CONTAINER_RESHAPE block container wide reshapes
(like changing disk numbers in arrays).
3. IMSM container_content handler is to load mdinfo for all volumes
and set both blocking flags in array.state field in mdinfo of
unsupported volumes. In case of some errors, all volumes can be
affected. Only blocked array is not activated (also reshaped as
result). The container wide reshapes are also blocked since by
metadata definition they require modifications of both arrays.
4. Incremental_container and Assemble functions check array.state and
do not activate volumes with blocking bits set.
5. assemble_container_content is changed to check container wide reshapes
before activating reshapes of assembled containers.
6. Grow_reshape and Grow_continue_command checks blocking bits
before starting reshapes or continueing (-G --continue) reshapes.
7. kill-subarray ignores array.state info and can remove requested array.
Signed-off-by: Marcin Labun <marcin.labun@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
So far there were 2 reshape continuation cases:
1. array is started /e.g. reshape was already invoked during initrd
start-up stage using "--freeze-reshape" option/
2. array is not started yet /"normal" assembling array under reshape case/
This patch narrows continuation cases in to single one. To do this
array should be started /set readonly in to array_state/ before calling
Grow_continue() function.
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
Reshape can be run for monitored arrays only /external metadata case/.
Before reshape can be executed, make sure that just starter array/container
is monitored. If not, run mdmon for it.
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
When container is assembled while reshape is active on one of its member
whole container can be required to be blocked from monitoring.
For such purpose field recovery blocked is added to mdinfo structure.
When metadata handler finds active reshape in container it should set
recovery_blocked field to disable whole container monitoring during
reshape.
For arrays that doesn't use containers, recovery_blocked field
has the same value as reshape_active field e.g. super0/1.
In fact,recovery is blocked during reshape for such arrays.
For ddf, metadata handler doesn't set reshape_active field,
so recovery_blocked is not set also.
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
During initrd phase continuing reshape will cause file system context
lost. This blocks ability to control reshape using checkpoints.
To avoid this, during initrd phase assemble has to be executed with
'--freeze-reshape' option. This causes that mdadm restores reshape
critical section only.
Reshape can be continued later after system full boot.
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
Reshape backup should be able to be restored during reshape continuation
also. To reuse already existing code it is moved to function.
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
For fdlist pointer allocated in assemble_container_content() function,
free() is never called. This patch fixes this memory leak.
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
The 'best' array only has 'bestcnt' entries allocated, so 'i' should
always be "< bestcnt", not "<= bestcnt".
Reported-by: "Lawrence, Joe" <Joe.Lawrence@stratus.com>
Signed-off-by: NeilBrown <neilb@suse.de>
Patch introduces support for reshape process restart for external metadata
using metadata specific data handling methods.
It introduces recover_backup() function that restores array to stable state
It is equivalent to Grow_restart() functionality for native metadata.
Signed-off-by: Maciej Trela <maciej.trela@intel.com>
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
Signed-off-by: Krzysztof Wojcik <krzysztof.wojcik@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
Some code currently clears 'info' before calling getinfo_super,
some code doesn't.
To be consistent, change it so no caller ever clears 'info',
but ever getinfo_super function must clear it.
Note that ->raid_disk may be meaningful if that 'map' is passed
non-NULL. In that case it is copied out before the structure
is zeroed.
Signed-off-by: NeilBrown <neilb@suse.de>
When array is in reshape state raid_disks field contains final disks number.
To know how many disks were added, disk.raid_disk index has to be compared
against old disk number computed using delta_disks.
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
If a degraded dirty array has some superblocks which are clean and
others that are dirty, and the dirty ones are newer by precisely '1'
in the event count, then the current code to force the array to be
clean will not work.
We need to make sure to find a superblock with most recent event count
and force that one to be 'clean'.
Reported-by: A J Wyborny <ajwyborny@gmail.com>
Signed-off-by: NeilBrown <neilb@suse.de>
When for ping_monitor() input devnum2devname() is used,
received string pointer should be passed to free() for memory release.
It is not made in several places. This use case should have function
to avoid memory leak.
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
Currently whole array geometry is set in sysfs_set_array(),
so none of disks (even for expansion) should fail during sysfs_add_disk()
Due to this expansion counter should be used for reshaped array when
disk slot is bigger than number of disks in array.
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
When array under reshape is assembled it has to be disabled from
monitoring as soon as possible. It can occur that this is i.e second
array in container and mdmon is loaded already.
Lack of blocking monitoring can cause change array state to active,
and reshape continuation will be not possible.
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
As containers can now grow, we need to use both Grow_restart (to
replay any backup-file) and Grow_continue when assembling the content
of a container.
Note that we don't pass a backup-file when doing incremental assembly.
If such is needed in that case, the assembly will fail.
To restart such arrays, explicit assembly is required.
Signed-off-by: NeilBrown <neilb@suse.de>
assemble_container_content() cannot close mdfd handle, as it could be
required by reshape continuation.
mdfd handle is closed outside this function, when it is not longer
necessary.
Call to Grow_continue is added for reshape continuation after
assembly.
In the nearest future, simple condition:
if (content->reshape_active)
before Grow_continue() call will be replaced by check function
for support container operation /reshape/.
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
During expansion there is more working disks that array can have.
Disks with set raid_disk (not a spare disk) during reshape should be counted
to allow array state transition to read_only state.
Array reconfiguration to new geometry should be done before reshape will
be started.
Signed-off-by: Adam Kwolek <adam.kwolek@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
When the metadata doesn't identify which array a spare belongs to
we normally require an explicit domain match to connect a spare
with an array.
However when the spare is explicitly listed in argv, it should be
safe to include as long as there is no domain conflict.
Signed-off-by: NeilBrown <neilb@suse.de>
Sometime we will need to know the difference between no domains found
and domains didn't match.
So allow domain_test to return different values and fix up all callers
to maintain current behaviour.
Signed-off-by: NeilBrown <neilb@suse.de>
If we find a device that has not superblock, we currently fail
unless in auto_assem mode.
However we really should only fail if the device was explicitly listed
in the arg list. So add a test for that.
Signed-off-by: NeilBrown <neilb@suse.de>
When there are any arrays in config file the spares with
domain not matching any array are not assembled because
auto assembly is not attempted.
Addition of ARRAY line with uuid=0:0:0:0 in config will work
with modified condition for gathering spares.
Signed-off-by: Anna Czarnowska <anna.czarnowska@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
If we find spares but no members of given array
we create container with just spares.
This allows auto assemble to pick up all lose imsm spares when there
is no config file.
When there is a valid config file and any array is assembled from it
we don't try auto assembly so we will not assemble spares that don't
match any array.
To remedy this we must add
ARRAY metadata=imsm UUID=00000000:00000000:00000000:00000000
to config file.
This container will include all remaining spares.
Signed-off-by: Anna Czarnowska <anna.czarnowska@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
Policy must be read on all disks identified as array members
to get array's domains list.
Currently it is only read on first array member in auto assembly mode.
Signed-off-by: Anna Czarnowska <anna.czarnowska@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
Imsm spare will only be taken if it matches domain of
identified members of currently assembled array.
This implies that:
- spare with null domain will match first array assembled.
- if array has null domain then no spare will match
If we allow spares to set st they may block assembly of subarrays.
This is because in auto-assembly tmpdev->used=0 for a spare not matching
any array. If we find such spare before container and set st, the content
will not get assembled.
We allow uuid_zero match any uuid in assembly as unsuitable spares will
be rejected on domain check.
Signed-off-by: Anna Czarnowska <anna.czarnowska@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
We need to refuse to assemble an arrays with bad blocks.
Initially there was condition in container_content function
that returns error value in the case when metadata store information
about bad blocks.
When the container_content function is called from functions NOT connected
with assemble (Kill_subarray, Detail) we get faulty error return value.
Patch introduces new flag in array.status - MD_SB_BBM_ERRORS. It is set
in container_content when bad blocks are detected and can be checked by
container_content caller.
Signed-off-by: Krzysztof Wojcik <krzysztof.wojcik@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
Though not having the proper backup file can cause data corruption, it
is not enough to justify not being able to start the array at all.
So allow "--invalid-backup" to be specified which says "just continue
even if a backup cannot be restored".
Signed-off-by: NeilBrown <neilb@suse.de>
An attempt to invoke super_by_fd() on device that has
metadata_version="none" always matches super0 (as test_version is "").
In Assemble() it results in segfault when load_container is invoked
(=null for super0).
As of now load_container is only started if it points to valid pointer.
Signed-off-by: Przemyslaw Czarnowski <przemyslaw.hawrylewicz.czarnowski@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
We shouldn't call remove_partitions until we have made a really firm
decision to include the device into the array.
Signed-off-by: NeilBrown <neilb@suse.de>
Incremental assembly works on such an array because the kernel sees the
disk as in-sync and that the array is reshaping. Teach Assemble() the
same assumptions.
This is only needed on kernels that do not initialize ->recovery_offset
when activating spares for reshape.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
When checking that a container matches the required uuid,
we need to call 'getinfo_super' before we have a 'content'
to test.
Reported-by: "Czarnowska, Anna" <anna.czarnowska@intel.com>
Signed-off-by: NeilBrown <neilb@suse.de>
Separate the load_container call from the load_super call,
and use different validity tests as appropriate.
Add some general code tidying and a bit of indent change to make
structure a little clearer.
Signed-off-by: NeilBrown <neilb@suse.de>
This is somewhat inconsistent with the last member of a
container getting special handling.
Just simplify it so the code seems to make sense and important
is easy to follow.
Signed-of-by: NeilBrown <neilb@suse.de>
Now that the next_member loop is much smaller it is easy to
just use 'content' rather than stashing it in 'tmpdev->content'.
So we can remove the 'content' field from 'struct mddev_dev'.
Signed-off-by: NeilBrown <neilb@suse.de>
We have a 'goto next_member' loop which is rather spread-out and
confusing.
Recent refactoring make it possible to contract that loop
significantly.
Signed-off-by: NeilBrown <neilb@suse.de>
In assemble, we see (inside a 'for' loop):
if (condition) {
lots of stuff
} else
something
small thing
if (same condition) {
lots more stuff
break;
}
where 'condition' cannot be changed in the middle.
So simplify this to
if (condition) {
lots of stuff
small thing
lots more stuff
break;
}
something
small thing
which duplicates the small thing, but provides much
conceptual simplicity.
Signed-off-by: NeilBrown <neilb@suse.de>
Remove the _t pointer typedef and remove the _s suffix for the
structure,
These things do not help readability.
Signed-off-by: NeilBrown <neilb@suse.de>
If we get an early error (e.g. not a block device) we need to
not continue through and check e.g. uuid.
Also make sure we set used=2 whenever we find an error, and don't
bother with ->free_super as 'goto loop' does that.
Now that we abort earlier, we can remove lots of tests on
tst && tst->sb
Signed-off-by: NeilBrown <neilb@suse.de>
This allows the info for a single array to be extracted,
so we don't have to write it into st->subarray.
For consistency, implement container_content for super0 and super1,
to just return the mdinfo for the single array.
Signed-off-by: NeilBrown <neilb@suse.de>
If a device - typically in a mirrored set - is assembled independently
of the other devices, and then attempted to be brought back into the
set it could contain inconsistent data. It should not be included.
So detect this situation by ensuring that the 'most recent' device is
believed to be active by every other device. If a device is wayward,
it will only consider fellow wayward devices to be active and will
think all others are failed or missing.
This patch only fixes --assemble, not --incremental
Signed-off-by: NeilBrown <neilb@suse.de>
To accurately detect when an array has been split and is now being
recombined, we need to track which other devices each thinks is
working.
We should never include a device in an array if it thinks that the
primary device has failed.
This patch just allows get_info_super to return a list of devices
and whether they are thought to be working or not.
Signed-off-by: NeilBrown <neilb@suse.de>
If an --update is requested by the relevant metadata doesn't
understand it, print a useful message rather than silently ignoring
the issue.
Signed-off-by: NeilBrown <neilb@suse.de>
To support incorpating a new bare device into a collection of arrays -
one partition each - mdadm needs a modest understanding of partition
tables.
The main needs to be able to recognise a partition table on one device
and copy it onto another.
This will be done using pseudo metadata types 'mbr' and 'gpt'.
Signed-off-by: NeilBrown <neilb@suse.de>
When we find a device that was recently part of the array but is now
out of date (based on the event count) we might want to add it back in
(like --re-add) if the likely cause was a connection problem or we
might not if the likely cause was device failure.
So make this a policy issue: if action=re-add or better, try to re-add
any device that looks like it might be part of the array.
This applies:
when we assemble the array: old devices will be evicted by the
kernel and need to be re-added.
when we assemble the array during --incr for the same reason.
when we find a device that could be added to a running array.
This doesn't affect arrays with external metadata at all.
For such arrays:
When the container is assembled, the most recent instance of each
device is included without reference to whether it is too old or not.
Then the metadata handler must which slices of which devices to
include in which array and with what state. So the
->container_content should probably check the policy and compare the
sequence numbers/event counts.
When a device is added (--add) to a container with active arrays
we only add as a 'spare'. --re-add doesn't seem to be an option.
When a device is added with -I ->container_content gets another
chance to assess things again. So again it should check the policy.
Signed-off-by: NeilBrown <neilb@suse.de>
commit 1ff9833928
broke the checking of metadata types via the 'auto' line.
Be moving 'load_super" before "conf_test_metadata" we left
tst->sb set even if conf_test_metadata fails, so the device will
actually be accepted and used.
So if we decide to reject the device, free the superblock so it is
clear that it is rejected.
Signed-off-by: NeilBrown <neilb@suse.de>
Found during testing:
- cannot check metadata for homehost before loading metadata.
- As 1.x metadata can has a state 'rebuilding' between
'spare' and 'ok', we need to include that in our calculations.
Signed-off-by: NeilBrown <neilb@suse.de>
If we find we cannot add the requested bitmap file when
assembling the array, then make sure to clean up properly
and don't leave a half-configured array.
Signed-off-by: NeilBrown <neilb@suse.de>
If --assemble is given a container and some other devices to assemble
an array from, it complains with an error because that doesn't make
sense.
However it currently also complains if the list of devices was extract
from the config file rather than being given on the command line.
That is not appropriate.
So add an '&& inargv' test to ensure that we are really complaining
about the right thing.
Signed-off-by: NeilBrown <neilb@suse.de>
Acked-by: Dan Williams <dan.j.williams@intel.com>
1.x metadata allows a device to be a member of the array while it
is still recoverying. So it is a working member, but is not
completely in-sync.
mdadm/assemble does not understand this distinction and assumes that a
work member is fully in-sync for the purpose of determining if there
are enough in-sync devices for the array to be functional.
So collect the 'recovery_start' value from the metadata and use it in
assemble when determining how useful a given device is.
Reported-by: Mikael Abrahamsson <swmike@swm.pp.se>
Signed-off-by: NeilBrown <neilb@suse.de>
Once load_super has succeeded, it should continue to succeed. However
devices can disappear etc so it is prudent to always check the return
status of load_super.
Signed-off-by: NeilBrown <neilb@suse.de>
Previously such things did not exist: ACTIVE and SYNC were either both
set or both clear. Recent changes with reshape means that a device
can be ACTIVE but not yet fully in-sync, so they need to be handled
and included in the array as active devices.
Signed-off-by: NeilBrown <neilb@suse.de>
When disks have conflicting container memberships (same container ids
but incompatible member arrays) --update=uuid can be used to move
offenders to a new container id by changing 'orig_family_num'.
Note that this only supports random updates of the uuid as the actual
uuid is synthesized. We also need to communicate the new
'orig_family_num' value to all disks involved in the update. A new
field 'update_private' is added to struct mdinfo to allow this
information to be transmitted.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
If an array name contains a "hostname:" prefix, then
--assemble will tend to leave it there, while --incremental
will strip it off (when chosing a device name during auto-assembly).
Make this more consistent: strip the name off if we decide that
the name will be treated as 'local'. Leave it on if it will be
treated as 'foreign'.
Signed-off-by: NeilBrown <neilb@suse.de>
If mdadm.conf contains
HOMEHOST <ignore>
or commandline contains
--homehost=<ignore>
then the check that array metadata mentions the given homehost is
replace by a check that the name recorded in the metadata is not
already used by some other array mentioned in mdadm.conf.
This allows more arrays to use their native name rather than having
an _NN suffix added.
This should only be used during boot time if all arrays required for
normal boot are listed in mdadm.conf.
If auto-assembly is used to find all array during boot, then the
HOMEHOST feature should be used to ensure there is no room for
confusion in choosing array names, and so it should not be set
to <ignore>.
Signed-off-by: NeilBrown <neilb@suse.de>
For container= and member= to be effective in an mdadm.conf line
they must both be present. So when checking for their absence we
need container != NULL || member != NULL.
Signed-off-by: NeilBrown <neilb@suse.de>
The line 'auto' in mdadm.conf can be used to disable assembly
of specific metadata types, or of all arrays.
This does not affect assembly of arrays listed in mdadm.conf
or on command line.
auto -all
will disable all auto-assembly.
auto -ddf
will cause mdadm to ignore ddf arrays that are not explicitly
mentioned, and auto assemble anything else it finds.
Signed-off-by: NeilBrown <neilb@suse.de>
If an array is created with --homehost=any, then --assemble and
--incremental will treat it as being local to 'this' host, no matter
what the name of this host is.
This is useful for array that will be given unique names and be
moved between machines.
This needs to be documented.
Signed-off-by: NeilBrown <neilb@suse.de>
When building container members with -IR, we need to ensure that
devices added to an active array preserve the 'in_sync' status so they
don't needlessly get rebuilt.
So allow sysfs_add_disk to do this (only works in kernels since
2.6.30) and pass the relevant flag down.
Signed-off-by: NeilBrown <neilb@suse.de>
wait not only for the name to appear, but for it to refer to the
correct device.
Sometimes old symlinks left lying around can be confusing.
Signed-off-by: NeilBrown <neilb@suse.de>
If we are assembling an array in a container and it isn't complete
enough to start yet, then
- don't start mdmon
- don't say the array is started
- don't wait for the device to appear in /dev
Signed-off-by: NeilBrown <neilb@suse.de>
Use mddev_busy() as GET_ARRAY_INFO can succeed on 'clear' arrays.
Ran into this after an encountering a case where mdadm -Ss ended in
segfault (missing check for NULL return from map_by_devnum() in
sles11:Manage.c). So, tried to stop the array by hand with echo clear >
md/array_state, after which I could not reassemble since GET_ARRAY_INFO
was succeeding.
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
Reshape with large chunk size can require a large stripe_cache.
We make this work when starting the reshape but not when
restarting at assemble time. So fix that.
Signed-off-by: NeilBrown <neilb@suse.de>
This is only significant for --assemble --force where some old
devices might be included into the array. If anything looks like
it isn't clean, the kernel will not allow a degraded array to be started.
Signed-off-by: NeilBrown <neilb@suse.de>
If any superblocks in a confused array had an event count of 0,
"mdadm -Af" would not update the event counts to assemble the array.
I don't remember why that text is there, and it has caused at least
one situation to be difficult to recover from. So remove the
test. --force means --force!
Signed-off-by: NeilBrown <neilb@suse.de>