The following is a modified version of the RDB r.q. I have given the RDB the ability to publish its results which I would like to then post-process in a q subscriber. In the background I have the ‘vanilla’ tick.q running. First of all, is this efficient, for I haven’t seen any examples with r.q having .u.pub? If not, what’s the recommended approach?

The memory usage of the q process running the RDB suddenly jumps from a baseline 1.5GB to 38.5GB at which point the process often crashes. The baseline RAM usage is consistent with the size of the data in the tick.q log file, but the latter value is both puzzling and concerning. I’m not entirely sure what’s causing this behaviour. A possible reason is that my subscriber is ‘slow’ but I’ve examined the time per iteration therein and that’s about 100ms –well below the timeout here of 1000ms.

/q tick/r.q [host]:port[:usr:pwd] [host]:port[:usr:pwd]
/2008.09.09 .k ->.q

l tick/u.q
l tick/sym.q

if[not "w"=first string .z.o;system "sleep 1"];

upd:insert;

/ get the ticker plant and history ports, defaults are 5010,5012
.u.x:.z.x,(count .z.x)_(":5010";":5012");

/ end of day: save, clear, hdb reload
.u.end:{t:tables`.;t@:where `g=attr each t@:`sym;.Q.hdpf[`$":",.u.x 1;`:.;x;`sym];@[;`sym;`g#] each t;};

/ init schema and sync up from log file;cd to hdb(so client save can run)
.u.rep:{(.[;();:;].)each x;if[null first y;:()];-11!y;system "cd ",1_-10_string first reverse y};
/ HARDCODE cd if other than logdir/db

/ connect to ticker plant for (schema;(logcount;log))
.u.rep .(hopen `$":",.u.x 0)"(.u.sub[`;`];`.u `i`L)";

/ modified section
.u.init[]

.z.ts:{.u.pub[`quote;quote]};
t 1000

p 5011