diff --git a/docs/dev/images/live_indexing_end_to_end.png b/docs/dev/images/live_indexing_end_to_end.png new file mode 100644 index 0000000..35547db Binary files /dev/null and b/docs/dev/images/live_indexing_end_to_end.png differ diff --git a/docs/dev/indexing.rst b/docs/dev/indexing.rst index ce7b390..a670c07 100644 --- a/docs/dev/indexing.rst +++ b/docs/dev/indexing.rst @@ -93,3 +93,193 @@ minutes a dynamical master takes indexes several phases the same way a GPU does Frames may be NumPy, torch or CuPy arrays; a GPU array is read in place through DLPack. Orientations and strains come back in kikuchipy's sample frame; turning them into a scan's is the caller's, with :mod:`de_ebsd.core.frames`. + +The design goal is a cost per buffer that is fixed and known in advance: every +step below is a batched operation over the whole buffer, and nothing loops over +patterns or phases. Only each pattern's phase, orientation and score -- and its +strain, when asked for -- leave the device. + +.. figure:: images/live_indexing_end_to_end.png + :alt: One pattern through the live indexer, from the frame to its orientation + and strain, then the scan's maps. + :width: 100% + + The live indexer end to end, on de-twin's duplex steel (20 kV, 10 degree + detector tilt). (1) The frame, divided by the static background and its own + background removed. (2) One product against every dictionary entry; the best + is the seed. (3) The 2 degree level, then the 1 degree level: the start and + one step either way about x, y and z, each scored, and a parabola an axis + moving to its peak. (4) One Gauss-Newton step on the three rotations: the + residual against the simulation, the derivative of the simulation about each + axis, and the step they solve for; every simulation is projected from the + master. (5) The same fit with the five deviatoric strains, on a pattern + simulated with a known strain and noise (the twin's steel has none): the + eight numbers fitted against those put in, and the polar decomposition into + the lattice rotation and the Biot strain. (6) Every pattern of the scan the + same way: the fitted bands, and the IPF, phase and score maps. + +Preparing the patterns +---------------------- + +:meth:`~de_ebsd.index.LiveIndexer.prepare` turns a buffer of raw frames into the +rows that are scored, in three steps: + +1. Each frame is averaged over the whole sensor down to the refine shape + (``refine_shape``, 128 x 128 by default). Averaging the whole sensor, rather + than cropping, keeps every band and every count. +2. It is divided by the scan's **static background**, the running mean of the + frames the scan has seen (:meth:`~de_ebsd.index.LiveIndexer.observe`). A live + scan's frames come from a few grains at a time, so the mean waits for 32 + frames before it is used; :meth:`~de_ebsd.index.LiveIndexer.restart_static` + starts it from frames chosen from all over the specimen (a survey's), which + average their bands away in fewer. On a single crystal every frame carries + the same bands, so the mean would keep them: there it is blurred. +3. The pattern's own slowly varying **dynamic background** is removed, the rows + are normalized to zero mean and unit length, and a copy is averaged again to + the search shape (``search_shape``, 32 x 32 by default). + +Whatever of the grains' bands the static background still holds -- the +**ghost** -- is tracked from the orientations already fitted, so the strain fit +below can take it out of patterns and simulations alike. + +Searching the dictionary +------------------------ + +Each phase's master is projected at the search shape over a sample of its +fundamental zone, at the dictionary's angular step: about 30,000 entries for a +cubic phase at 3 degrees. Every phase's entries are stacked into one matrix, so +:meth:`~de_ebsd.index.LiveIndexer.search` is one matrix product of the buffer +against that matrix: the best entry gives the phase and a seed orientation +together, and the score is the normalized cross-correlation. A low-symmetry +phase's dictionary is scored 65,536 entries at a time, keeping each pattern's +best so far, so the scores never outgrow one chunk. + +The search's copy of the masters is blurred by 2 degrees on the sphere. A sharp +dynamical pattern's score falls so steeply away from its orientation that at a +3-degree step no entry may be near enough, and a distant wrong one wins; on a +twin scan 6-9 % of the patterns landed 40-60 degrees off unblurred, none at +2 degrees. The search only has to land in the right basin -- the refine does the +rest. + +Refining the orientation +------------------------ + +:meth:`~de_ebsd.index.LiveIndexer.refine` builds a local dictionary around each +seed, projected from the (unblurred) master on the fly at the refine shape: the +seed and the six rotations of one step about each axis. A parabola through each +axis's three scores moves the orientation, and the step halves each level, from +2 degrees down to 0.12. Every level is the same seven projections per pattern, +so its cost is fixed. With CuPy on a CUDA device the levels run as one fused +kernel launch, one block per pattern, and the levels of a degree and up are +scored at half the refine resolution: a step that size moves the bands by +several pixels, which a quarter of them resolves as well. + +With ``fit=True``, :meth:`~de_ebsd.index.LiveIndexer.index` fits the three +rotations on from the refine's result by Gauss-Newton on the normalized +pattern, with the Jacobian taken from the master's gradient. The stepped levels +stop a little short on a noisy pattern; the fit takes them the rest of the way. +An indexer that will fit is built with +:func:`~de_ebsd.index.live.fitted_refine_steps_deg`, which stops the stepped +levels at 1 degree where the fit is fused: the fit does what the finer levels +did, for less. + +Calibrating the pattern center +------------------------------ + +:meth:`~de_ebsd.index.LiveIndexer.refine_pattern_center` refines the center +shared by a buffer the same way the orientation is refined: seven candidate +centers per level -- the current one and one step either way in x, y and z -- +each scored by re-refining every pattern's orientation there, with a parabola +per coordinate and the step halving from 0.008 to 0.001 of the detector. It +uses the buffer's best-scored patterns, and refines their orientations far finer +than indexing does (down to 0.03 degrees): a center 0.004 off moves each +pattern's best orientation by a few tenths of a degree, which coarser steps +follow only in jumps, leaving the mean score flat and the center stuck short. + +Live strain +=========== + +A strained lattice changes the pattern in ways an orientation cannot: the bands +move relative to one another. There are two ways to measure it live: fitting +each pattern against a dynamical master, or correlating it against a real +reference pattern. + +Fitted against the master +------------------------- + +:meth:`~de_ebsd.index.LiveIndexer.refine_deformation` (and +:meth:`~de_ebsd.index.LiveIndexer.index_with_strain`, which indexes and then +calls it) fits each pattern's orientation and strain together. A crystal +deformed by ``F`` shows the master's pattern at ``g F^-1 r`` for a ray ``r``, so +a candidate is the matrix ``g (I - beta)``, with ``beta`` eight trace-free +numbers: three rotations and five deviatoric strains. The fused kernels project +that matrix like any orientation. Each Gauss-Newton iteration projects the +current fit and its 16 central differences and solves the 8 x 8 normal +equations, for every pattern at once. + +- **The hydrostatic strain is not measured.** A gnomonic projection cannot see a + uniform dilation, so the ninth component is left out; the strain returned is + the symmetric, trace-free Biot strain in the sample frame, with the lattice + rotation split off by a polar decomposition. +- **The rotations are fitted with the strains** because a strain left out of the + model is partly fitted as a turn: fitting both is also what sharpens the + orientations. +- **Each step is bounded.** A step solves a local linear model, and on a poor + pattern -- a grain boundary, two patterns overlapping -- that model can point + far outside the elastic range. Without a limit on each step, 1-3 % of a real + scan's fits ran off to strains of 0.1 and more. A fit that ends scoring below + its seed is refused: its strain is NaN. +- **A line search** tries multiples of each Gauss-Newton step, because a plain + step closes only about half of what is left (the simulation is sharper than + the pattern). Three iterations with it did what seven plain ones did; on a + 15 kV silicon indent the four strongest strains reached 0.94-1.05 of + CrossCourt's HR-EBSD values. + +On CUDA with CuPy the fit instead runs a fixed schedule of passes in one launch, +one block per pattern, keeping the best-scored point it visits: about 9 ms per +256 patterns on a Titan X, against about 42 ms for the torch fit. + +**Related work.** Fitting a deformation to the whole pattern by Gauss-Newton is +the integrated digital image correlation of Vermeij and Hoefnagels [VH2018]_, +there against a reference pattern; fitting the pattern center, the orientation +and the absolute strain against dynamically simulated patterns is Tanaka and +Wilkinson's pattern matching [TW2019]_. This fit is the first against the +second: a Gauss-Newton fit of the orientation and the deviatoric strain against +the master, batched over every pattern on the GPU, with a bounded step and a +line search. + +.. [VH2018] T. Vermeij and J. P. M. Hoefnagels, "A consistent full-field + integrated DIC framework for HR-EBSD", Ultramicroscopy (2018). +.. [TW2019] T. Tanaka and A. J. Wilkinson, "Pattern matching analysis of electron + backscatter diffraction patterns for pattern centre, crystal orientation and + absolute elastic strain determination: accuracy and precision assessment", + Ultramicroscopy (2019), doi:10.1016/j.ultramic.2019.04.006. + +**The pattern center must come from the geometry.** On a flat screen, moving the +center is exactly a linear map of the ray directions, so within one pattern a +center error is indistinguishable from a strain. Each pattern may be given its +own center (:meth:`~de_ebsd.index.LiveIndexer.center_maps`), but that center has +to come from where the beam was on the specimen and the detector geometry -- a +center fitted to each pattern on its own would absorb the very strain being +measured. + +Against a reference pattern +--------------------------- + +:meth:`~de_ebsd.index.LiveIndexer.strain_from_reference` is HR-EBSD [WMD2006]_: +it cross-correlates regions of each pattern against one real reference frame +(``hrebsd``) and solves for the strain and lattice rotation relative to it. No +master enters, so how well the simulation matches the camera does not limit it. +In exchange it measures only differences from the reference, which should +itself be strain-free, and only within the reference's grain: the remap holds +for lattice rotations of a few degrees. + +On a twin scan with a random 3e-3 strain at a 128 x 128 refine, the error +relative to a reference in each grain was 1.7e-3, against 7.3e-4 absolute from +the fit against the master; correlating regions against simulations instead was +6e-3. The fit against the master measures absolute strain everywhere it indexes; +a reference is the independent check where a strain-free region is known. + +.. [WMD2006] A. J. Wilkinson, G. Meaden and D. J. Dingley, "High-resolution + elastic strain measurement from electron backscatter diffraction patterns: + new levels of sensitivity", Ultramicroscopy (2006). diff --git a/src/de_ebsd/index/live/deformation.py b/src/de_ebsd/index/live/deformation.py index 6136671..b5a7dbc 100644 --- a/src/de_ebsd/index/live/deformation.py +++ b/src/de_ebsd/index/live/deformation.py @@ -141,6 +141,12 @@ def refine_deformation( Fitting the rotations alongside the strains is what sharpens the orientations: a strain left out of the model is partly fitted as a turn. + The fit is Vermeij and Hoefnagels' integrated DIC (a Gauss-Newton fit of a + deformation to the whole pattern; Ultramicroscopy, 2018) taken against + simulated patterns, as Tanaka and Wilkinson's pattern matching does for the + pattern center, orientation and absolute strain (Ultramicroscopy, 2019, + doi:10.1016/j.ultramic.2019.04.006); see ``docs/dev/indexing.rst``. + *g* should already be within a few tenths of a degree (:meth:`LiveIndexer.refine`). *center_maps* (:meth:`center_maps`) give each pattern its own center. *ghost* (:attr:`ghost`) is taken out of the patterns and the simulations diff --git a/upcoming_changes/+live-indexing-figure.doc.rst b/upcoming_changes/+live-indexing-figure.doc.rst new file mode 100644 index 0000000..4aa6eb2 --- /dev/null +++ b/upcoming_changes/+live-indexing-figure.doc.rst @@ -0,0 +1 @@ +The live-indexing guide has a figure of the indexer end to end (prepare, search, the 2 and 1 degree levels, the Gauss-Newton fit, the strain and its split, the scan's maps), and cites the work its strain fit and reference strain follow: Vermeij and Hoefnagels' integrated DIC, Tanaka and Wilkinson's pattern matching, and Wilkinson, Meaden and Dingley's HR-EBSD.