bug: no error when env is missing

[cyrus-za]

Node.js version

18.16.0

NPM version

pnpm 8.7.4

@checkly/cli version

checkly 4.1.0

Steps to reproduce

Run npm create checkly

Using the defaults, create a check in checks and let it point to a spec file of any basic playwright test

create a lib/env.ts file with the following content

const { BASE_URL } = process.env

if(!BASE_URL) throw new Error("BASE_URL is a required env")

Create .env file with BASE_URL=https://github.com

edit your .spec file to import BASE_URL

import { BASE_URL } from './lib/env'

test.describe("my test suite", () => {
    test("my test", async ({ page }) => {
      await page.goTo(BASE_URL)
      await expect(page).toHaveTitle(/Github/)
  })
})

Run the check with npx checkly test --env-file .env --record
Your test should pass without issues

Run npx checkly deploy -f

What is expected?

Your check should run on schedule and succeed

What is actually happening?

Your check now runs on schedule but it immediately fails.
Logs show no error message other than "No tests found"

If you try to edit the test and run it manually within checkly dashboard you also get no logs at all.

Any additional comments?

After spending a few hours debugging, I realised it was because when I do npx checkly deploy it does not use the .env file (duh!) and I had to manually add the env in the checkly dashboard

It would be really helpful if the error is actually logged