"If a party is lost, the DM may choose the direction the party moves in, or use a random die roll. The DM must keep track of the party's actual position, as well as the direction the party believes it is moving. For example, the DM determines that a party ... has become lost. ... the party wishes to travel north; however, the DM has secretly determined that the party will head northeast. If after travelling in this direction for 6 miles, should the group decided to turn northwest, they will actually turn north." ("Travelling in the Wilderness", Dungeons and Dragons Expert Rules, p.X56)
A diagram using hexes follows.
I have never used this rule.
I did invoke a "becoming lost" rule once, in my Jungles of Chult game, which in 5e is of course a skill check (a Survival [Wisdom] check, DC 10 for waterways or DC 15 for jungles). Failure indicates that when the party moves into the "next" hex, roll a d6 to randomly determine which neighboring hex they actually enter. "... While the party is lost, players can't pinpoint the group's location on their map of Chult. The next time a navigator succeeds on a Wisdom (Survival) check made to navigate, reveal the party's actual location to the players." ("Chapter 2: The Land of Chult", Tomb of Annihilation, p.38)
Actually, I didn't use that rule by-the-text either, because I allowed a player character to make a second roll to get on track after the navigators (River Mist and Flask of Wine) got themselves turned around with a failed roll.
Where I have used rules like it is when playing Outdoor Survival (the hex map of which, as I understand it, was the basis for much D&D exploration back in the genesis-days). In that game it makes sense. As a board game, each player has perfect information about the board and the state of his "character" and position on the board, and even everyone else's character and position. The only way to simulate "lostness" in such a case is to force a player to move in a random direction as indicated by a die roll. (Every turn you roll a die to determine how lost you are; a good roll allows you to choose a direction, a bad roll forces you to move in a die-rolled direction)
|image from Boardgamegeek|
But D&D is emphatically not a game of perfect information. A player has the information on his character sheet and on whatever map he may have drawn, and beyond that relies on the referee's description of his character's location and surroundings. D&D is a game of discovery, of epistemology one might say ... a game in which players map a space according to the referee's descriptions which may or may not conform to the referee's map of the "actual" space, and where such accuracy or deviation can have serious consequences for the state of the characters and the game.
So D&D doesn't need to simulate "lostness" in the way that Outdoor Survival does. The players already don't know the map, and don't know where they're going. And I don't much like the idea of running "getting lost" at the table by travelling in a random direction--to go back to the B/X quote at the top, to keep track of where the characters are on my map and also where they think they are on my map, and to adjust their movements accordingly seems like a hassle. I won't lie, I already have trouble keeping track of where the party is on a hex map just for myself when I'm supposed to be notating it in pencil ... sometimes I forget to mark a day's travel, or when they're not moving at the speed of one hex per day I forget how far through the hex they've moved this turn ... (maybe I'll post about hex maps in the future).
Long story short, I have a thought for another way to simulate becoming lost that would mitigate the hassle of the map and confusion on the referee's (my) part:
When moving from hex to hex (or from area to area, maybe), treat navigation as an "open door" type check. The party rolls a die (be it a straight 1-2 on a d6 or rolling a d20 plus Survival [Wisdom] against some DC, or whatever) and if they succeed, they may proceed into the next area. If they fail the roll, their day's travel is suspended as they try to find a passable way into the next area but their way is blocked by impassable brambles, a raging river, a sheer cliff, or their own incompetence at wilderness travel. They do not advance or move anywhere else.
On the next day, they may either try again with another roll, or must try to circumvent the barrier through magic or by traveling another direction (depending on whether you, the referee, treat the roll as final or as re-rollable).
In most situations, the party is able to backtrack the way they've come without having to roll anything, especially in the cast that they've mapped their way there. But in especially thick wilderness (a jungle, say), the party might have to make a roll to make their way into another hex or area even if that is the same hex or area from whence they've just come. I'd probably still allow a bonus to backtrack, especially with a map ...
This method of getting lost would cut out an element of the game that I feel is intellectually interesting, but which often bogs down at the table: the mode of play in which it is up to the players to ensure that their map is correct, to interrogate the fictive space that the referee is describing to be certain that they have grasped the "truth" of their representation of it.
Theoretically, the rules for getting lost in B/X D&D actually appeal to me, in the way that the sliding corridors, one-way doors, and secret teleporters that Gygax seems so fond of in dungeons are theoretically appealing. What fun it would be, I think to myself reading such rules, to be in a dungeon where I was suddenly deposited somewhere from which I don't know how to get to the exit. I would have to map myself back to a familiar space on my map!
The problem with this type of play is that most of the time I'm stuck with two or three hours for the game at most, and I've got to resolve things by the end (for my pick-up games ... same-people campaigns are a bit different, and you can put things on hold until next session). Jeff Rients' Triple Secret Random Dungeon Fate Chart of Very Probable Doom (scroll to the bottom) can help with resolving what happens when characters are stuck or lost in the dungeon (or wilderness--need a new table!) at the end of a session; but what happens when characters are split up in the middle of play? Do you split time between them? Cut one side out and have them roll to get out while you stay with the other group?
I know there must be good ways for referees to deal with these things effectively and without bogging play down, but I am not at the moment privy to them. Perhaps someone out there has procedures that make this part of the game as easy as the rest is--and I would be glad to learn from them!--but as of now, I haven't played in anyone else's hexcrawl, and only one other referee's "squarecrawl", so I'm just muddling things out for myself.