Head Above The Clouds

If you have spent any length of time having to process large numbers of records in MarkLogic, then you must have already experienced the dreadful XMDP-EXPNTREECACHEFULL error message. This is simply, but mercilessly, telling you that there is not enough memory in your system to process your query and generate the final result(s). Most of the times, it is the result of poor code techniques and/or lack of indexing on the relevant elements/attributes. The simple method below has proven useful on many occasions and is rather straightforward to implement. Basically, instead of accumulating the processing of all records in memory, you are handing the processing of each record to a separate thread and letting the Task Server spawn them all and return the result(s). In this example, I am assuming you want to update the dataset in MarkLogic. Explaining the code: efficiently identify the records to process - lines 1 to 9 process each record separately - line 11 use xdmp:spawn-function() for e...

(Praia do Porto, Imbituba, circa 2014) Much have changed in Imbituba , Santa Catarina, Brazil, from the time when I first surfed there. I have not seen any of the changes. The last time I was there was in the late 70's. I know about them from reading online articles very, very occasionally. Perhaps the most important change was the construction of a jetty which I can only assume wasn't good for surfing. In any case, I have no desire to talk about the Imbituba of today. This is a trip back to the past, during the second half of the seventies. For a young men from Rio without much money for trips, the option was to take a 13-hour bus ride and then rent a very cheap fisherman's hut at the main beach road. No asphalt.  The bus trip was unpleasant. Not because of the people. They were friendly. But the comfort in the bus was very lacking. Today it would have been a nightmare to attem...