Summary
The Results Instance Manifest field transfers only ever records transfers
declared through the old, per-instance transfer-mode metadata directive.
Transfers declared the newer way, through the named-paths group
definition.json descriptor, do actually run, but are never recorded in
transfers, or anywhere else in the manifest. A csvpath whose transfers are
all defined via the group descriptor will show an empty transfers field
even though real transfers happened during the run.
Where the field is set
csvpath/managers/results/result_registrar.py, register_complete:
if self.result.csvpath.transfers:
tpaths = self.result.csvpath.csvpaths.results_manager.transfers_manager.transfer_paths(
self.result
)
mdata.transfers = tpaths
self.result.csvpath.transfers (csvpath/csvpath.py:861-863) reads only
the transfer-mode key from that one csvpath own leading-comment metadata
(csvpath/modes/transfer_mode.py). And
TransfersManager.transfer_paths() (csvpath/managers/results/transfers_manager.py:167-175)
is built on exactly that same source:
def transfer_paths(self, result) -> list[tuple[str, str, str, str, str]]:
transfers = result.csvpath.transfers
return self._transfer_paths(result, transfers)
What is missing
TransfersManager.do_transfers_if(result)
(csvpath/managers/results/transfers_manager.py:28-48) actually calls two
independent mechanisms for every result:
def do_transfers_if(self, result) -> None:
try:
self.do_transfer_mode_if(result)
except Exception:
...
try:
self.do_description_transfers_if(result)
except Exception:
...
do_transfer_mode_if is the old, per-instance transfer-mode path, the
same one that feeds mdata.transfers.
do_description_transfers_if (lines 50-74) is the newer mechanism, sourced
from the named-paths group descriptor:
def do_description_transfers_if(self, result) -> None:
csvpaths = self.results_manager.csvpaths
pathsmgr = csvpaths.paths_manager
name = result.paths_name
describer = pathsmgr.describer
transfers = describer.get_transfers(name)
if transfers and transfers.path_transfers is not None:
n = result.identity_or_index
if n is not None and n in transfers.path_transfers:
self.do_description_transfers(result, name, transfers.path_transfers.get(n))
This walks into do_description_transfers and do_description_transfer
(lines 76-98), which build their own tpaths and call _do_transfers
directly. Those transfers really execute, files really get copied, but
nothing on this path ever writes to mdata.transfers, or to any other
manifest field. The group descriptor mechanism also supports several
distinct trigger states (on_complete_all, on_complete_valid, and an
error-triggered branch), none of which are distinguished, or even present,
in the manifest today.
Impact
Anyone reading a Results Instance Manifest to find out what transfers ran
for that csvpath only sees the old-style, per-instance ones. Any group,
descriptor-based transfers are invisible in the manifest even though they
happened. This will also matter for the in-progress references v3 work,
where a :transfers() style accessor would currently only be able to expose
half the picture.
Suggested fix direction
Make mdata.transfers collect the union of both mechanisms results:
transfer_paths(result) from do_transfer_mode_if, plus whatever
do_description_transfers/do_description_transfer actually copied from
the descriptor path. Likely means having do_description_transfer return
or accumulate its tpaths the same way do_transfer_mode_if already does,
so do_transfers_if can merge both lists before handing them to whichever
registrar sets mdata.transfers. Worth deciding at the same time whether
the trigger state, on_complete_all vs on_complete_valid vs error, should
be recorded per transfer in the manifest, since that information is
currently dropped entirely.
Found while reviewing manifest fields for the references v3 work.
Summary
The Results Instance Manifest field
transfersonly ever records transfersdeclared through the old, per-instance
transfer-modemetadata directive.Transfers declared the newer way, through the named-paths group
definition.jsondescriptor, do actually run, but are never recorded intransfers, or anywhere else in the manifest. A csvpath whose transfers areall defined via the group descriptor will show an empty
transfersfieldeven though real transfers happened during the run.
Where the field is set
csvpath/managers/results/result_registrar.py,register_complete:self.result.csvpath.transfers(csvpath/csvpath.py:861-863) reads onlythe
transfer-modekey from that one csvpath own leading-comment metadata(
csvpath/modes/transfer_mode.py). AndTransfersManager.transfer_paths()(csvpath/managers/results/transfers_manager.py:167-175)is built on exactly that same source:
What is missing
TransfersManager.do_transfers_if(result)(
csvpath/managers/results/transfers_manager.py:28-48) actually calls twoindependent mechanisms for every result:
do_transfer_mode_ifis the old, per-instancetransfer-modepath, thesame one that feeds
mdata.transfers.do_description_transfers_if(lines 50-74) is the newer mechanism, sourcedfrom the named-paths group descriptor:
This walks into
do_description_transfersanddo_description_transfer(lines 76-98), which build their own
tpathsand call_do_transfersdirectly. Those transfers really execute, files really get copied, but
nothing on this path ever writes to
mdata.transfers, or to any othermanifest field. The group descriptor mechanism also supports several
distinct trigger states (
on_complete_all,on_complete_valid, and anerror-triggered branch), none of which are distinguished, or even present,
in the manifest today.
Impact
Anyone reading a Results Instance Manifest to find out what transfers ran
for that csvpath only sees the old-style, per-instance ones. Any group,
descriptor-based transfers are invisible in the manifest even though they
happened. This will also matter for the in-progress references v3 work,
where a
:transfers()style accessor would currently only be able to exposehalf the picture.
Suggested fix direction
Make
mdata.transferscollect the union of both mechanisms results:transfer_paths(result)fromdo_transfer_mode_if, plus whateverdo_description_transfers/do_description_transferactually copied fromthe descriptor path. Likely means having
do_description_transferreturnor accumulate its
tpathsthe same waydo_transfer_mode_ifalready does,so
do_transfers_ifcan merge both lists before handing them to whicheverregistrar sets
mdata.transfers. Worth deciding at the same time whetherthe trigger state,
on_complete_allvson_complete_validvs error, shouldbe recorded per transfer in the manifest, since that information is
currently dropped entirely.
Found while reviewing manifest fields for the references v3 work.