Head Above The Clouds

MarkLogic has an extremely fast search engine but it can be adversely affected by inefficient queries and the absence of element/attribute indexing. Here is an example which makes use of a very efficient API function called cts:value-tuples that searches on indexed elements and attributes. Explaining the code: utility function to transform results to CSV format (lines 5-13) fetch journal titles for a given year (lines 15-24) iterate through the years and journal titles (lines 26-27) get all records for a given journal for a given year (lines 29-34) build references for fetching journal title, search term and publication year (lines 36-40) return frequency of a given search term per journal per year using cts:value-tuples (lines 42-46) Note how results are output by frequency in descending order and transformed into CSV format.   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 xquery versi

If you have spent any length of time having to process large numbers of records in MarkLogic, then you must have already experienced the dreadful XMDP-EXPNTREECACHEFULL error message. This is simply, but mercilessly, telling you that there is not enough memory in your system to process your query and generate the final result(s). Most of the times, it is the result of poor code techniques and/or lack of indexing on the relevant elements/attributes. The simple method below has proven useful on many occasions and is rather straightforward to implement. Basically, instead of accumulating the processing of all records in memory, you are handing the processing of each record to a separate thread and letting the Task Server spawn them all and return the result(s). In this example, I am assuming you want to update the dataset in MarkLogic. Explaining the code: efficiently identify the records to process - lines 1 to 9 process each record separately - line 11 use xdmp:spawn-function() for e

(Praia do Porto, Imbituba, circa 2014) Much have changed in Imbituba , Santa Catarina, Brazil, from the time when I first surfed there. I have not seen any of the changes. The last time I was there was in the late 70's. I know about them from reading online articles very, very occasionally. Perhaps the most important change was the construction of a jetty which I can only assume wasn't good for surfing. In any case, I have no desire to talk about the Imbituba of today. This is a trip back to the past, during the second half of the seventies. For a young men from Rio without much money for trips, the option was to take a 13-hour bus ride and then rent a very cheap fisherman's hut at the main beach road. No asphalt.  The bus trip was unpleasant. Not because of the people. They were friendly. But the comfort in the bus was very lacking. Today it would have been a nightmare to attem

The Loss of the Human Self in 21st Century America. What is better...to have free will in a controlled world or to be controlled in a free world? To most of us this will sound like a false dichotomy. Nobody is controlling me you say. And the world is too large and complex to be controlled anyways. I really wished both notions were true. But I see mounting evidence to the contrary. Not only this evidence is piling up, it is intensifying. Oh, I can smell a conspiracy theory coming up you say. And your reticence will be taken as entirely appropriate skepticism since the mere mention of a conspiracy theory will dissipate any suspicions that it might actually exist. Funny how this works. It is an unexamined statement that effectively discourages examination. It is a double wham for the near irrational. But why is it so effective? Have you yourself never "conspired" with anyone to make something happen? It could have been something that you and

...LIVE EVERY MINUTE MORE INTENTIONALLY . Note that I am not saying more intensely. That is not realistic, at least for most of us. Intentions are more within our continuous grasp than intensity. And what do I mean exactly by more intentionally? Consider more carefully what you are saying and particularly what you are doing. And if at all possible, do a lot of listening. And not just to the people you deem worthy. Even big fools do things which make sense to them. Sometimes understanding the sense behind it can prove instructive when interacting with lesser fools. ...KEEP IN MIND THAT ELOQUENCE AND TRUTH ARE NOT THE SAME . This is much harder to do than one thinks. Much harder...particularly when you are in the midst of a situation when someone is being very eloquent and has already persuaded most of the people around you. And if EVERYONE around you becomes persuaded, it gets nearly impossible. But you must withstand your ground, particularly if sai

This is an example of a class in IronPython that can upload a file to a web site. This is a port from the example given in http://www.codeproject.com/KB/cs/uploadfileex.aspx , except that I did not consider the use of cookies...but the addition should be trivial. Points to note are: the parameter query_string is of type NameValueCollection the parameter url must be a valid URI (e.g. www.somewebsite.com/somepage ) the parameter content_type can be 'multipart/form-data'. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 import System from System import * from System.Text import * from System.IO import * import clr clr . AddReference( "System.Xml" ) from System.Xml import * clr . AddReference( "System.Web" ) from System.Web import * clr . AddReference( &qu