feat(sitemap): add lastmod to extracted page output

[nikitachapovskii-dev]

Each page in the Sitemap Extractor output now carries a lastmod field with the page's last-modification time.

The lastmod is taken from two sources in order of preference:

The tag declared for the URL in the sitemap.
The Last-Modified response header, used only when the sitemap has no .
If neither is present, lastmod is null.

Output example

{
  "url": "https://example.com/page",
  "status": 200,
  "lastmod": "2024-01-01T00:00:00.000Z"
}

Closes #281

[nikitachapovskii-dev]

@nicklamonov please check readme

[nicklamonov]

Closes #281, not #301

[nicklamonov]

Rather "date-time of last modification."

[nicklamonov]

same as above

[nicklamonov]

Readme is ok (besides a small nit mentioned).

If you could build it for testing under some tag, I should be able to validate the functionality too.

Thank you!

[nikitachapovskii-dev]

@nicklamonov https://console.apify.com/admin/users/ZscMwFR5H7eCtWtyh/actors/rGeTNESChDZ65EbYh/source
please test under 0.0
it has a different branch, forked revive-monorepo + this exact changes so it can be built on platform

[ruocco-l]

I was about to say that in case there is no information about it we would end up with every item having null which is not really pleasing, and could be avoided by using undefined, which will not create the column in the dataset if no item has the property, but I guess printing null is still valid information and it does not leave you wonder in case you have it in one run and not in the others. LGTM 👍

[nicklamonov]

Small question:
Are we able to change the order of fields in the Output in Console?

Now it's like this:

1. lastmod
2. status
3. url

But ideally I'd reverse them, so they are displayed in following priority (like in json):

1. url
2. status
3. lastmod

Also, is it ok that we don't have lastmod in the dataset schema (in actor.json)? Is it because we may not poppulate it if all are nulls or was it just missed?

Otherwise, from functional perspective it looks good.

[nikitachapovskii-dev]

https://console.apify.com/view/runs/xcMCso8kBMsgGgmUK

cc @nicklamonov

[nicklamonov]

Great!
Thanks!
Looks good.