Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 10 additions & 28 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ void setSize(uint16_t w, uint16_t h)
uint16_t tilesNeeded(uint16_t w, uint16_t h)
```

This returns the number of tiles required to cache the given map size.
This returns the -most pessimistic- number of tiles required to cache the given map size.

### Resize the tiles cache

Expand All @@ -95,12 +95,16 @@ Use the above `tilesNeeded` function to calculate a safe and sane cache size if
### Fetch a map

```c++
bool fetchMap(LGFX_Sprite &map, double longitude, double latitude, uint8_t zoom)
bool fetchMap(LGFX_Sprite &map, double longitude, double latitude, uint8_t zoom, unsigned long timeoutMS = 0)
```

- Overflowing `longitude` are wrapped and normalized to +-180°.
- Overflowing `latitude` are clamped to +-90°.
- Valid range for the `zoom` level is from `getMinZoom()` to `getMaxZoom()`.
- `timeoutMS` can be used to throttle the amount of downloaded tiles per call.
Setting it to anything other than `0` sets a timeout. Sane values start around ~100ms.
**Note:** No more tiles will be downloaded after the timeout expires, but tiles that are downloading will be finished.
**Note:** You might end up with missing map tiles. Or no map at all if you set the timeout too short.

### Free the psram memory used by the tile cache

Expand Down Expand Up @@ -133,40 +137,18 @@ const int numberOfProviders = OSM_TILEPROVIDERS;

**Note:** In the default setup there is only one provider defined.

See `src/TileProvider.hpp` for example setups for [https://www.thunderforest.com/](https://www.thunderforest.com/) that only require an API key and commenting/uncommenting 2 lines.

Registration and a hobby tier are available for free.

### Adding tile providers

Other providers should work if a new definition is created in `src/TileProvider.hpp`.
Check out the existing templates to see how this works.

If you encounter a problem or want to request support for a new provider, please check the [issue tracker](../../issues) for existing reports or [open an issue](../../issues/new).


### Get the provider name

```c++
char *getProviderName()
```

### Set the render mode

```c++
void setRenderMode(RenderMode mode)
```

Available modes:
## Adding tile providers

- `RenderMode::ACCURATE` (default)
Downloads map tiles **without a timeout**, ensuring a complete map with **no missing tiles** in most cases.
Best suited for reliability and full-quality rendering.
See `src/TileProvider.hpp` for example setups for [https://www.thunderforest.com/](https://www.thunderforest.com/) that only require you to register for a **free** API key and adjusting/uncommenting 2 lines in the config.
Register for a ThunderForest free tier [here](https://manage.thunderforest.com/users/sign_up?price=hobby-project-usd) without needing a creditcard to sign up.

- `RenderMode::FAST`
Downloads map tiles **with a timeout**.
This mode can produce the map **more quickly**, but some **tiles may be missing** if a request times out.
Ideal when operating under time constraints.
If you encounter a problem or want to request support for a new provider, please check the [issue tracker](../../issues) for existing reports or [open an issue](../../issues/new).

## Example code

Expand Down
48 changes: 38 additions & 10 deletions src/OpenStreetMap-esp32.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -253,6 +253,7 @@ void OpenStreetMap::runJobs(const std::vector<TileJob> &jobs)
log_d("submitting %i jobs", (int)jobs.size());

pendingJobs.store(jobs.size());
startJobsMS = millis();
for (const TileJob &job : jobs)
if (xQueueSend(jobQueue, &job, portMAX_DELAY) != pdPASS)
{
Expand Down Expand Up @@ -298,7 +299,7 @@ bool OpenStreetMap::composeMap(LGFX_Sprite &mapSprite, TileBufferList &tilePoint
return true;
}

bool OpenStreetMap::fetchMap(LGFX_Sprite &mapSprite, double longitude, double latitude, uint8_t zoom)
bool OpenStreetMap::fetchMap(LGFX_Sprite &mapSprite, double longitude, double latitude, uint8_t zoom, unsigned long timeoutMS)
{
if (!tasksStarted && !startTileWorkerTasks())
{
Expand Down Expand Up @@ -335,6 +336,7 @@ bool OpenStreetMap::fetchMap(LGFX_Sprite &mapSprite, double longitude, double la
return false;
}

mapTimeoutMS = timeoutMS;
TileBufferList tilePointers;
updateCache(requiredTiles, zoom, tilePointers);
if (!composeMap(mapSprite, tilePointers))
Expand All @@ -351,7 +353,7 @@ void OpenStreetMap::PNGDraw(PNGDRAW *pDraw)
getPNGCurrentCore()->getLineAsRGB565(pDraw, destRow, PNG_RGB565_BIG_ENDIAN, 0xffffffff);
}

bool OpenStreetMap::fetchTile(ReusableTileFetcher &fetcher, CachedTile &tile, uint32_t x, uint32_t y, uint8_t zoom, String &result)
bool OpenStreetMap::fetchTile(ReusableTileFetcher &fetcher, CachedTile &tile, uint32_t x, uint32_t y, uint8_t zoom, String &result, unsigned long timeout)
{
String url = currentProvider->urlTemplate;
url.replace("{x}", String(x));
Expand All @@ -360,7 +362,7 @@ bool OpenStreetMap::fetchTile(ReusableTileFetcher &fetcher, CachedTile &tile, ui
if (currentProvider->requiresApiKey && strstr(url.c_str(), "{apiKey}"))
url.replace("{apiKey}", currentProvider->apiKey);

MemoryBuffer buffer = fetcher.fetchToBuffer(url, result, renderMode);
MemoryBuffer buffer = fetcher.fetchToBuffer(url, result, timeout);
if (!buffer.isAllocated())
return false;

Expand Down Expand Up @@ -406,18 +408,37 @@ void OpenStreetMap::tileFetcherTask(void *param)
if (job.z == 255)
break;

const uint32_t elapsedMS = millis() - osm->startJobsMS;
if (osm->mapTimeoutMS && elapsedMS >= osm->mapTimeoutMS)
{
log_w("Map timeout (%lu ms) exceeded after %lu ms, dropping job",
osm->mapTimeoutMS, elapsedMS);

osm->invalidateTile(job.tile);
--osm->pendingJobs;
continue;
}

// compute remaining time budget for this job
uint32_t remainingMS = 0;
if (osm->mapTimeoutMS > 0)
{
remainingMS = osm->mapTimeoutMS - elapsedMS;
if (remainingMS == 0)
remainingMS = 1; // minimum non-zero
}

String result;
if (!osm->fetchTile(fetcher, *job.tile, job.x, job.y, job.z, result))
if (!osm->fetchTile(fetcher, *job.tile, job.x, job.y, job.z, result, remainingMS))
{
log_e("Tile fetch failed: %s", result.c_str());
job.tile->valid = false;
const size_t tileByteCount = osm->currentProvider->tileSize * osm->currentProvider->tileSize * 2;
memset(job.tile->buffer, 0, tileByteCount);
osm->invalidateTile(job.tile);
}
else
{
job.tile->valid = true;
log_d("core %i fetched tile z=%u x=%lu, y=%lu in %lu ms", xPortGetCoreID(), job.z, job.x, job.y, millis() - startMS);
log_d("core %i fetched tile z=%u x=%lu, y=%lu in %lu ms",
xPortGetCoreID(), job.z, job.x, job.y, millis() - startMS);
}
job.tile->busy = false;
--osm->pendingJobs;
Expand Down Expand Up @@ -496,7 +517,14 @@ bool OpenStreetMap::setTileProvider(int index)
return true;
}

void OpenStreetMap::setRenderMode(RenderMode mode)
void OpenStreetMap::invalidateTile(CachedTile *tile)
{
renderMode = mode;
if (!tile)
return;

const size_t tileByteCount = currentProvider->tileSize * currentProvider->tileSize * 2;
memset(tile->buffer, 0, tileByteCount);

tile->valid = false;
tile->busy = false;
}
12 changes: 6 additions & 6 deletions src/OpenStreetMap-esp32.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -38,10 +38,8 @@
#include "MemoryBuffer.hpp"
#include "ReusableTileFetcher.hpp"
#include "fonts/DejaVu9-modded.h"
#include "RenderMode.hpp"

constexpr uint16_t OSM_BGCOLOR = lgfx::color565(32, 32, 128);
constexpr uint16_t OSM_TILE_TIMEOUT_MS = 1000;
constexpr UBaseType_t OSM_TASK_PRIORITY = 1;
constexpr uint32_t OSM_TASK_STACKSIZE = 5120;
constexpr uint32_t OSM_JOB_QUEUE_SIZE = 50;
Expand Down Expand Up @@ -91,10 +89,9 @@ class OpenStreetMap
void setSize(uint16_t w, uint16_t h);
uint16_t tilesNeeded(uint16_t mapWidth, uint16_t mapHeight);
bool resizeTilesCache(uint16_t numberOfTiles);
bool fetchMap(LGFX_Sprite &sprite, double longitude, double latitude, uint8_t zoom);
bool fetchMap(LGFX_Sprite &sprite, double longitude, double latitude, uint8_t zoom, unsigned long timeoutMS = 0);
inline void freeTilesCache();

void setRenderMode(RenderMode mode);
bool setTileProvider(int index);
const char *getProviderName() { return currentProvider->name; };
int getMinZoom() const { return currentProvider->minZoom; };
Expand All @@ -110,14 +107,14 @@ class OpenStreetMap
void runJobs(const std::vector<TileJob> &jobs);
CachedTile *findUnusedTile(const tileList &requiredTiles, uint8_t zoom);
CachedTile *isTileCached(uint32_t x, uint32_t y, uint8_t z);
bool fetchTile(ReusableTileFetcher &fetcher, CachedTile &tile, uint32_t x, uint32_t y, uint8_t zoom, String &result);
bool fetchTile(ReusableTileFetcher &fetcher, CachedTile &tile, uint32_t x, uint32_t y, uint8_t zoom, String &result, unsigned long timeoutMS);
bool composeMap(LGFX_Sprite &mapSprite, TileBufferList &tilePointers);
static void tileFetcherTask(void *param);
static void PNGDraw(PNGDRAW *pDraw);
void invalidateTile(CachedTile *tile);

static inline thread_local OpenStreetMap *currentInstance = nullptr;
static inline thread_local uint16_t *currentTileBuffer = nullptr;
RenderMode renderMode = RenderMode::ACCURATE;
const TileProvider *currentProvider = &tileProviders[0];
std::vector<CachedTile> tilesCache;

Expand All @@ -127,6 +124,9 @@ class OpenStreetMap
std::atomic<int> pendingJobs = 0;
bool tasksStarted = false;

unsigned long mapTimeoutMS = 0; // 0 means no timeout
unsigned long startJobsMS = 0;

uint16_t mapWidth = 320;
uint16_t mapHeight = 240;

Expand Down
7 changes: 0 additions & 7 deletions src/RenderMode.hpp

This file was deleted.

52 changes: 30 additions & 22 deletions src/ReusableTileFetcher.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@

#include "ReusableTileFetcher.hpp"

ReusableTileFetcher::ReusableTileFetcher() { renderMode = RenderMode::ACCURATE; }
ReusableTileFetcher::ReusableTileFetcher() {}
ReusableTileFetcher::~ReusableTileFetcher() { disconnect(); }

void ReusableTileFetcher::sendHttpRequest(const String &host, const String &path)
Expand All @@ -42,9 +42,8 @@ void ReusableTileFetcher::disconnect()
currentPort = 80;
}

MemoryBuffer ReusableTileFetcher::fetchToBuffer(const String &url, String &result, RenderMode mode)
MemoryBuffer ReusableTileFetcher::fetchToBuffer(const String &url, String &result, unsigned long timeoutMS)
{
renderMode = mode;
String host, path;
uint16_t port;
if (!parseUrl(url, host, path, port))
Expand All @@ -53,22 +52,28 @@ MemoryBuffer ReusableTileFetcher::fetchToBuffer(const String &url, String &resul
return MemoryBuffer::empty();
}

if (!ensureConnection(host, port, result))
if (!ensureConnection(host, port, timeoutMS, result))
return MemoryBuffer::empty();

sendHttpRequest(host, path);
size_t contentLength = 0;
if (!readHttpHeaders(contentLength, result))
if (!readHttpHeaders(contentLength, timeoutMS, result))
return MemoryBuffer::empty();

if (contentLength == 0)
{
result = "Empty response (Content-Length=0)";
return MemoryBuffer::empty();
}

auto buffer = MemoryBuffer(contentLength);
if (!buffer.isAllocated())
{
result = "Buffer allocation failed";
return MemoryBuffer::empty();
}

if (!readBody(buffer, contentLength, result))
if (!readBody(buffer, contentLength, timeoutMS, result))
return MemoryBuffer::empty();

return buffer;
Expand All @@ -93,33 +98,38 @@ bool ReusableTileFetcher::parseUrl(const String &url, String &host, String &path
return true;
}

bool ReusableTileFetcher::ensureConnection(const String &host, uint16_t port, String &result)
bool ReusableTileFetcher::ensureConnection(const String &host, uint16_t port, unsigned long timeoutMS, String &result)
{
if (!client.connected() || host != currentHost || port != currentPort)
{
disconnect();
client.setConnectionTimeout(renderMode == RenderMode::FAST ? 100 : 5000);
if (!client.connect(host.c_str(), port, renderMode == RenderMode::FAST ? 100 : 5000))

// If caller didn’t set a timeout, fall back to 5000ms
uint32_t connectTimeout = timeoutMS > 0 ? timeoutMS : OSM_DEFAULT_TIMEOUT_MS;
if (!client.connect(host.c_str(), port, connectTimeout))
{
result = "Connection failed to " + host;
return false;
}
currentHost = host;
currentPort = port;
log_i("(Re)connected on core %i", xPortGetCoreID());
log_i("(Re)connected on core %i (timeout=%lu ms)", xPortGetCoreID(), connectTimeout);
}
return true;
}

bool ReusableTileFetcher::readHttpHeaders(size_t &contentLength, String &result)
bool ReusableTileFetcher::readHttpHeaders(size_t &contentLength, unsigned long timeoutMS, String &result)
{
String line;
line.reserve(OSM_MAX_HEADERLENGTH);
contentLength = 0;
bool start = true;

uint32_t headerTimeout = timeoutMS > 0 ? timeoutMS : OSM_DEFAULT_TIMEOUT_MS;

while (client.connected())
{
if (!readLineWithTimeout(line, renderMode == RenderMode::FAST ? 300 : 5000))
if (!readLineWithTimeout(line, headerTimeout))
{
result = "Header timeout";
disconnect();
Expand All @@ -129,7 +139,7 @@ bool ReusableTileFetcher::readHttpHeaders(size_t &contentLength, String &result)
line.trim();
if (start)
{
if (!line.startsWith("HTTP/1.1"))
if (!line.startsWith("HTTP/1."))
{
result = "Bad HTTP response: " + line;
disconnect();
Expand All @@ -150,30 +160,28 @@ bool ReusableTileFetcher::readHttpHeaders(size_t &contentLength, String &result)
}

if (contentLength == 0)
{
result = "Missing or invalid Content-Length";
disconnect();
return false;
}
log_w("Content-Length = 0 (valid empty body)");

return true;
}

bool ReusableTileFetcher::readBody(MemoryBuffer &buffer, size_t contentLength, String &result)
bool ReusableTileFetcher::readBody(MemoryBuffer &buffer, size_t contentLength, unsigned long timeoutMS, String &result)
{
uint8_t *dest = buffer.get();
size_t readSize = 0;
unsigned long lastReadTime = millis();
const unsigned long timeoutMs = (renderMode == RenderMode::FAST) ? 300 : 5000;

// Respect caller’s remaining budget, default to 5000ms if none
const unsigned long maxStall = timeoutMS > 0 ? timeoutMS : OSM_DEFAULT_TIMEOUT_MS;

while (readSize < contentLength)
{
size_t availableData = client.available();
if (availableData == 0)
{
if (millis() - lastReadTime >= timeoutMs)
if (millis() - lastReadTime >= maxStall)
{
result = "Timeout: " + String(timeoutMs) + " ms";
result = "Body read stalled for " + String(maxStall) + " ms";
disconnect();
return false;
}
Expand Down
Loading