Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added docs/dev/images/live_indexing_end_to_end.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
190 changes: 190 additions & 0 deletions docs/dev/indexing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -93,3 +93,193 @@ minutes a dynamical master takes indexes several phases the same way a GPU does
Frames may be NumPy, torch or CuPy arrays; a GPU array is read in place through
DLPack. Orientations and strains come back in kikuchipy's sample frame; turning
them into a scan's is the caller's, with :mod:`de_ebsd.core.frames`.

The design goal is a cost per buffer that is fixed and known in advance: every
step below is a batched operation over the whole buffer, and nothing loops over
patterns or phases. Only each pattern's phase, orientation and score -- and its
strain, when asked for -- leave the device.

.. figure:: images/live_indexing_end_to_end.png
:alt: One pattern through the live indexer, from the frame to its orientation
and strain, then the scan's maps.
:width: 100%

The live indexer end to end, on de-twin's duplex steel (20 kV, 10 degree
detector tilt). (1) The frame, divided by the static background and its own
background removed. (2) One product against every dictionary entry; the best
is the seed. (3) The 2 degree level, then the 1 degree level: the start and
one step either way about x, y and z, each scored, and a parabola an axis
moving to its peak. (4) One Gauss-Newton step on the three rotations: the
residual against the simulation, the derivative of the simulation about each
axis, and the step they solve for; every simulation is projected from the
master. (5) The same fit with the five deviatoric strains, on a pattern
simulated with a known strain and noise (the twin's steel has none): the
eight numbers fitted against those put in, and the polar decomposition into
the lattice rotation and the Biot strain. (6) Every pattern of the scan the
same way: the fitted bands, and the IPF, phase and score maps.

Preparing the patterns
----------------------

:meth:`~de_ebsd.index.LiveIndexer.prepare` turns a buffer of raw frames into the
rows that are scored, in three steps:

1. Each frame is averaged over the whole sensor down to the refine shape
(``refine_shape``, 128 x 128 by default). Averaging the whole sensor, rather
than cropping, keeps every band and every count.
2. It is divided by the scan's **static background**, the running mean of the
frames the scan has seen (:meth:`~de_ebsd.index.LiveIndexer.observe`). A live
scan's frames come from a few grains at a time, so the mean waits for 32
frames before it is used; :meth:`~de_ebsd.index.LiveIndexer.restart_static`
starts it from frames chosen from all over the specimen (a survey's), which
average their bands away in fewer. On a single crystal every frame carries
the same bands, so the mean would keep them: there it is blurred.
3. The pattern's own slowly varying **dynamic background** is removed, the rows
are normalized to zero mean and unit length, and a copy is averaged again to
the search shape (``search_shape``, 32 x 32 by default).

Whatever of the grains' bands the static background still holds -- the
**ghost** -- is tracked from the orientations already fitted, so the strain fit
below can take it out of patterns and simulations alike.

Searching the dictionary
------------------------

Each phase's master is projected at the search shape over a sample of its
fundamental zone, at the dictionary's angular step: about 30,000 entries for a
cubic phase at 3 degrees. Every phase's entries are stacked into one matrix, so
:meth:`~de_ebsd.index.LiveIndexer.search` is one matrix product of the buffer
against that matrix: the best entry gives the phase and a seed orientation
together, and the score is the normalized cross-correlation. A low-symmetry
phase's dictionary is scored 65,536 entries at a time, keeping each pattern's
best so far, so the scores never outgrow one chunk.

The search's copy of the masters is blurred by 2 degrees on the sphere. A sharp
dynamical pattern's score falls so steeply away from its orientation that at a
3-degree step no entry may be near enough, and a distant wrong one wins; on a
twin scan 6-9 % of the patterns landed 40-60 degrees off unblurred, none at
2 degrees. The search only has to land in the right basin -- the refine does the
rest.

Refining the orientation
------------------------

:meth:`~de_ebsd.index.LiveIndexer.refine` builds a local dictionary around each
seed, projected from the (unblurred) master on the fly at the refine shape: the
seed and the six rotations of one step about each axis. A parabola through each
axis's three scores moves the orientation, and the step halves each level, from
2 degrees down to 0.12. Every level is the same seven projections per pattern,
so its cost is fixed. With CuPy on a CUDA device the levels run as one fused
kernel launch, one block per pattern, and the levels of a degree and up are
scored at half the refine resolution: a step that size moves the bands by
several pixels, which a quarter of them resolves as well.

With ``fit=True``, :meth:`~de_ebsd.index.LiveIndexer.index` fits the three
rotations on from the refine's result by Gauss-Newton on the normalized
pattern, with the Jacobian taken from the master's gradient. The stepped levels
stop a little short on a noisy pattern; the fit takes them the rest of the way.
An indexer that will fit is built with
:func:`~de_ebsd.index.live.fitted_refine_steps_deg`, which stops the stepped
levels at 1 degree where the fit is fused: the fit does what the finer levels
did, for less.

Calibrating the pattern center
------------------------------

:meth:`~de_ebsd.index.LiveIndexer.refine_pattern_center` refines the center
shared by a buffer the same way the orientation is refined: seven candidate
centers per level -- the current one and one step either way in x, y and z --
each scored by re-refining every pattern's orientation there, with a parabola
per coordinate and the step halving from 0.008 to 0.001 of the detector. It
uses the buffer's best-scored patterns, and refines their orientations far finer
than indexing does (down to 0.03 degrees): a center 0.004 off moves each
pattern's best orientation by a few tenths of a degree, which coarser steps
follow only in jumps, leaving the mean score flat and the center stuck short.

Live strain
===========

A strained lattice changes the pattern in ways an orientation cannot: the bands
move relative to one another. There are two ways to measure it live: fitting
each pattern against a dynamical master, or correlating it against a real
reference pattern.

Fitted against the master
-------------------------

:meth:`~de_ebsd.index.LiveIndexer.refine_deformation` (and
:meth:`~de_ebsd.index.LiveIndexer.index_with_strain`, which indexes and then
calls it) fits each pattern's orientation and strain together. A crystal
deformed by ``F`` shows the master's pattern at ``g F^-1 r`` for a ray ``r``, so
a candidate is the matrix ``g (I - beta)``, with ``beta`` eight trace-free
numbers: three rotations and five deviatoric strains. The fused kernels project
that matrix like any orientation. Each Gauss-Newton iteration projects the
current fit and its 16 central differences and solves the 8 x 8 normal
equations, for every pattern at once.

- **The hydrostatic strain is not measured.** A gnomonic projection cannot see a
uniform dilation, so the ninth component is left out; the strain returned is
the symmetric, trace-free Biot strain in the sample frame, with the lattice
rotation split off by a polar decomposition.
- **The rotations are fitted with the strains** because a strain left out of the
model is partly fitted as a turn: fitting both is also what sharpens the
orientations.
- **Each step is bounded.** A step solves a local linear model, and on a poor
pattern -- a grain boundary, two patterns overlapping -- that model can point
far outside the elastic range. Without a limit on each step, 1-3 % of a real
scan's fits ran off to strains of 0.1 and more. A fit that ends scoring below
its seed is refused: its strain is NaN.
- **A line search** tries multiples of each Gauss-Newton step, because a plain
step closes only about half of what is left (the simulation is sharper than
the pattern). Three iterations with it did what seven plain ones did; on a
15 kV silicon indent the four strongest strains reached 0.94-1.05 of
CrossCourt's HR-EBSD values.

On CUDA with CuPy the fit instead runs a fixed schedule of passes in one launch,
one block per pattern, keeping the best-scored point it visits: about 9 ms per
256 patterns on a Titan X, against about 42 ms for the torch fit.

**Related work.** Fitting a deformation to the whole pattern by Gauss-Newton is
the integrated digital image correlation of Vermeij and Hoefnagels [VH2018]_,
there against a reference pattern; fitting the pattern center, the orientation
and the absolute strain against dynamically simulated patterns is Tanaka and
Wilkinson's pattern matching [TW2019]_. This fit is the first against the
second: a Gauss-Newton fit of the orientation and the deviatoric strain against
the master, batched over every pattern on the GPU, with a bounded step and a
line search.

.. [VH2018] T. Vermeij and J. P. M. Hoefnagels, "A consistent full-field
integrated DIC framework for HR-EBSD", Ultramicroscopy (2018).
.. [TW2019] T. Tanaka and A. J. Wilkinson, "Pattern matching analysis of electron
backscatter diffraction patterns for pattern centre, crystal orientation and
absolute elastic strain determination: accuracy and precision assessment",
Ultramicroscopy (2019), doi:10.1016/j.ultramic.2019.04.006.

**The pattern center must come from the geometry.** On a flat screen, moving the
center is exactly a linear map of the ray directions, so within one pattern a
center error is indistinguishable from a strain. Each pattern may be given its
own center (:meth:`~de_ebsd.index.LiveIndexer.center_maps`), but that center has
to come from where the beam was on the specimen and the detector geometry -- a
center fitted to each pattern on its own would absorb the very strain being
measured.

Against a reference pattern
---------------------------

:meth:`~de_ebsd.index.LiveIndexer.strain_from_reference` is HR-EBSD [WMD2006]_:
it cross-correlates regions of each pattern against one real reference frame
(``hrebsd``) and solves for the strain and lattice rotation relative to it. No
master enters, so how well the simulation matches the camera does not limit it.
In exchange it measures only differences from the reference, which should
itself be strain-free, and only within the reference's grain: the remap holds
for lattice rotations of a few degrees.

On a twin scan with a random 3e-3 strain at a 128 x 128 refine, the error
relative to a reference in each grain was 1.7e-3, against 7.3e-4 absolute from
the fit against the master; correlating regions against simulations instead was
6e-3. The fit against the master measures absolute strain everywhere it indexes;
a reference is the independent check where a strain-free region is known.

.. [WMD2006] A. J. Wilkinson, G. Meaden and D. J. Dingley, "High-resolution
elastic strain measurement from electron backscatter diffraction patterns:
new levels of sensitivity", Ultramicroscopy (2006).
6 changes: 6 additions & 0 deletions src/de_ebsd/index/live/deformation.py
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,12 @@ def refine_deformation(
Fitting the rotations alongside the strains is what sharpens the
orientations: a strain left out of the model is partly fitted as a turn.

The fit is Vermeij and Hoefnagels' integrated DIC (a Gauss-Newton fit of a
deformation to the whole pattern; Ultramicroscopy, 2018) taken against
simulated patterns, as Tanaka and Wilkinson's pattern matching does for the
pattern center, orientation and absolute strain (Ultramicroscopy, 2019,
doi:10.1016/j.ultramic.2019.04.006); see ``docs/dev/indexing.rst``.

*g* should already be within a few tenths of a degree (:meth:`LiveIndexer.refine`).
*center_maps* (:meth:`center_maps`) give each pattern its own center.
*ghost* (:attr:`ghost`) is taken out of the patterns and the simulations
Expand Down
1 change: 1 addition & 0 deletions upcoming_changes/+live-indexing-figure.doc.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
The live-indexing guide has a figure of the indexer end to end (prepare, search, the 2 and 1 degree levels, the Gauss-Newton fit, the strain and its split, the scan's maps), and cites the work its strain fit and reference strain follow: Vermeij and Hoefnagels' integrated DIC, Tanaka and Wilkinson's pattern matching, and Wilkinson, Meaden and Dingley's HR-EBSD.
Loading