The precision one can achieve depends pretty much on the quality of the plot and of course on how careful the scan is performed. For a high quality plot and a carefully conducted scan the precision is realistically not better than the equivalent of approximately one pixel. To get a feeling on the achievable precision select View->Current Precision. The information dialog that pops up displays the current precision corresponding to +/- one pixel in every direction. For logarithmic axis the uncertainties will be of course asymmetric. Although not a perfect estimate, the values shown should give you a feeling on the precision of the scan and how serious you should take the obtained values. This is not applicable for heavily distorted plots.
Coordinates and measured/scanned values are displayed with in the 'g' format meaning either as plain decimal numbers [-]9.9 or in exponential form [-]9.9E[+|-]999, whichever is the most concise. Once the markers are all set, xyscan sets the "precision" of the displayed numbers to a reasonable value. In the exponential format the precision represents the number of digits after the decimal point. In the standard form the precision represents the maximum number of significant digits (trailing zeros are omitted). The number of significant digits can be changed with the Edit->Significant Digits menu item. If the Measure Tool is used and no markers are set the number of significant digits is fixed at 6.