[HOLD] [$250] [MEDIUM] Smartscan: Receipt thumbnail should show the top of the receipt

[m-natarajan]

If you haven’t already, check out our contributing guidelines for onboarding and email contributors@expensify.com to request to join our Slack channel!

---

Version Number: 1.4.22-6
Reproducible in staging?: yes
Reproducible in production?:.yes
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL:
Issue reported by: @quinthar
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1704757554079199

Action Performed:

1. create an IOU using receipt scanning
2. Observe the thumbnail of the receipt

Expected Result:

Should be showing from the top of the receipt

Actual Result:

Showing the middle portion of the receipt

Workaround:

unknown

Platforms:

Which of our officially supported platforms is this issue occurring on?

• Android: Native
• Android: mWeb Chrome
• iOS: Native
• iOS: mWeb Safari
• MacOS: Chrome / Safari
• MacOS: Desktop

Screenshots/Videos

Add any screenshot/video evidence

View all open jobs on GitHub

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~019bb6b442f89fba02
• Upwork Job ID: 1744585981661257728
• Last Price Increase: 2024-04-24
• Automatic offers:
• cubuspl42 | Reviewer | 28117185
• dukenv0307 | Contributor | 28117186

[melvin-bot]

Job added to Upwork: https://www.upwork.com/jobs/~019bb6b442f89fba02

[melvin-bot]

Triggered auto assignment to @adelekennedy (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

[melvin-bot]

Triggered auto assignment to Contributor-plus team member for initial proposal review - @cubuspl42 (External)

[dukenv0307]

Proposal

Please re-state the problem that we are trying to solve in this issue.

Showing the middle portion of the receipt

What is the root cause of that problem?

For resizeMode cover, the image will have default positioning which is in the middle. The behavior that we want is similar to the object-position: top in CSS, but this React Native doesn't support it out of the box so we need to add it to React Native.

What changes do you think we should make in order to solve the problem?

Make the View wrapping the image to have overflow hidden, for the image itself, it will take full width, variable height according to the aspectRatio (not fixed height).

The View wrapper already has overflow hidden here so all we have to do is to make sure our Image component can handle the object position top case.

1. Add a objectPositionTop boolean prop to our Image component to control this behavior (later we can modify to support other positions like bottom). I intended to make it as similar and reusable as possible (similar usage as native object-position: top in CSS), we just need to modify in common Image component and we can use it everywhere we use images in the app.
2. Add the prop to ImageWithSizeCalculation as well and pass it through to the inner Image, In here, add objectPositionTop because we want that behavior for ThumbnailImage. We can make it a prop of ThumbnailImage as well so we can control the position of ThumbnailImage.
3. Then handle in Image so the objectPositionTop property works
   Here

const [aspectRatio, setAspectRatio] = useState();

Here

RNImage.getSize(source.uri, (width, height) => {
    onLoad({nativeEvent: {width, height}});

    if (props.objectPositionTop) {
        setAspectRatio(height ? width / height : 'auto');
    }
});

Here

style={[forwardedProps.style, aspectRatio !== undefined && {aspectRatio, height: 'auto'}]}

For native platforms, we should do similarly in the index.native file, for native platforms, we won't get the image dimensions unless we load the image, we might want to use Image.getSize similar to on web to make sure it's independent from the image loading, and then we can prevent displaying of the image while the aspectRatio is not yet available, it might help prevent some potential issues.

We need to double check all possible cases of showing thumbnail receipt and add objectPositionTop to those places as needed.

What alternative solutions did you explore? (Optional)

Another approach is to translate the image vertically an amount such that instead of appearing from the center, image will appear at the top, this will require some math to calculate the translation so I prefer the aspectRatio approach above

I explored using object-position: top and background-position: top as well but react native does not yet support it.

[aswin-s]

Proposal

Please re-state the problem that we are trying to solve in this issue.

Receipt thumbnail is not showing the top of the image

What is the root cause of that problem?

For displaying a receipt <ThumbnailImage/> component is used with shouldDynamicallyResize prop set to false. This causes the image to fill to the given container width and height. However the ImageWithSizeCalculation component covers the image to available width and height. This ensures that the displayed image properly covers the parent container. However a side effect of cover is that the image gets scaled and positioned such that it entirely covers the container, there by causing the vertical or horizontal alignment of image to be adjusted as needed.

What changes do you think we should make in order to solve the problem?

For always displaying the top section of receipt thumbnail, the image container width needs to be set to the width of thumbnail and the height should be set as per aspect ratio of the image. This makes sure that the thumbnail image gets rendered fully and not get scaled or repositioned due to 'cover'.

function ThumbnailImage({previewSourceURL, style, isAuthTokenRequired, imageWidth = 200, imageHeight = 200, shouldDynamicallyResize = true}: ThumbnailImageProps) {
    // Use a variable to save container width on layout
    const [containerSize, setContainerSize] = useState({ width: 0, height: 0 });
    ...

    // Now compute the aspect ratio of the image
    const aspectRatio = currentImageWidth && currentImageHeight ? currentImageWidth / currentImageHeight : 1;
    // Compute aspect ratio of container
    const containerAspectRatio = containerSize.width / containerSize.height;
    
    // Get static image size as per aspect ratios
    const getImageSize = useCallback(() => {
        if (!aspectRatio) {
            return [styles.w100, styles.h100]
        }
        if (aspectRatio > containerAspectRatio) {
            return [{
                height: containerSize.height,
                width: containerSize.height * aspectRatio
            }]
        }
        return [{
            width: containerSize.width,
            height: containerSize.width / aspectRatio
        }]
    }, [aspectRatio, containerAspectRatio, containerSize.height, containerSize.width, styles.h100, styles.w100])

    const sizeStyles = shouldDynamicallyResize ? [StyleUtils.getWidthAndHeightStyle(currentImageWidth ?? 0, currentImageHeight)] : getImageSize();
 
    return (
        <View
            style={[style, styles.overflowHidden]}
	    // Get container width             
            onLayout={(e) => setContainerSize(e.nativeEvent.layout)}
        >
            <View style={[...sizeStyles, styles.alignItemsCenter, styles.justifyContentCenter]}>
                <ImageWithSizeCalculation
                    url={previewSourceURL}
                    onMeasure={updateImageSize}
                    isAuthTokenRequired={isAuthTokenRequired}
                />
            </View>
        </View>
    );

Above logic also handles images whose width is larger than height. Also the logic works for both native and web platforms.

There are a couple of screens like MoneyRequestConfirmationList and ReportActionItemImage etc where Image component is used instead of ThumbnailImage for displaying receipt. Such instances need to be replaced so that all occurrences of receipt image is properly displayed

Result

iOS

fix-ios_out.mp4

Web

fix-web_out.mp4

What alternative solutions did you explore? (Optional)

None (edited)

[dukenv0307]

@aswin-s FYI your solution's the same as one of my alternate solution

A potential alternative is to make the View wrapping the image to have overflow hidden, for the image itself, it will take full width, variable height according to the aspectRatio (not fixed height).

I'm writing the implementation details for my proposal and updating soon

[shubham1206agra]

Proposal

Please re-state the problem that we are trying to solve in this issue.

Receipt thumbnail is not showing the top of the image

What is the root cause of that problem?

Inside ReportActionItemImage, the ThumbnailImage component is being used. But this component uses 100% for height and width, which makes image center (looks like center but not exactly).

What changes do you think we should make in order to solve the problem?

We need to use aspectRatio technique to properly scale the images so that always top can be shown. We need to correctly scale the images for landscape, and portrait images.

Test branch - https://github.com/shubham1206agra/App/tree/test-thumbnail

Note - Naming of function is not final.

What alternative solutions did you explore? (Optional)

NA

[aswin-s]

Noted @dukenv0307. My proposal has just one approach detailed which works on all platforms. I've tested on both iOS and web and shared the screen grabs as well. I've also addressed scenarios like serving local image when receipt upload is in progress in the above video. So it is not exactly same or abstract like your proposal.

I'll leave the merit of the proposal to the C+.