Quantopian's community platform is shutting down. Please read this post for more information and download your code.
Back to Community
Logging more content while remaining under the logging limit
Blue Seahawk
edited Mar 10, 2019
Show Q Tools and Tips

A way to log a greater amount of info and stay under some of the logging limits.

There are a lot of other uses for it. This example is number of shares and symbol (alphabetically).
The main trick is to store strings with new lines (\n) and log each group before going over somewhere around 1024 bytes with each log.info().

Here you can see BCLI was long and is going short, the current state -4900 BCLI, compared to this message:
2016-08-01 13:00 WARN Your order for -8525 shares of BCLI has been partially filled. 5050 shares were successfully sold. 3475 shares were not filled by the end of day and were canceled.

def initialize(context):  
    schedule_function(positions, date_rules.every_day(), time_rules.market_close())

def positions(context, data):    # Show number of shares and symbol, staying under log limits  
    line       = ''  
    chunk      = '.\n'  # necessary to start on beginning of next line for alignment  
    line_len   = 90  
    buffer_len = 1000   # each group  
    sym_sids   = {}  
    for s in context.portfolio.positions.keys():    # to be able to sort by symbol  
        sym_sids[str(s.symbol)] = s  
    for sym in sorted(sym_sids.keys()):  
        s = sym_sids[sym]  
        line += '{} {}'.format(str(context.portfolio.positions[s].amount).rjust(7), s.symbol.ljust(6))  
        if len(line) > line_len:  
            chunk += '   ' + line + '\n'  
            line  = ''                    # start next line  
        if len(chunk) > buffer_len:  
            log.info(chunk)  
            chunk = '.\n'

    if len(chunk) > 2:       # if anything remaining  
        log.info(chunk)

Output. Notice all the same minute.

2016-08-01 12:59 positions:29 INFO .  
       600 AAL      1756 AAOI      436 ABAX     -834 ABTX     1434 ACAT    -1443 ACBI    -2002 ACLS  
      3972 ACTG    -1412 ADMS     5984 ADVM    -2505 ADXS    -1046 AEGN      263 AEPI      895 AFSI  
      1153 AGNC    -1394 AGTC     3759 AINV     2362 AKBA      168 ALGT      258 ALOG      803 AMAG  
     -3110 AMD       124 AMGN     3285 AMKR     1356 AMPH     1495 AMRI      379 AMSF      288 AMSG  
       710 AMTD      -27 AMZN      189 ANAT     -551 ANDE     1305 ANGO      418 ANIK      361 ANIP  
     -6899 ANTH      705 AOBC      774 APEI    -1713 APIC     -418 APOG    -2212 APOL    -2349 AQMS  
     -2531 AQXP    -1960 ARDX   -11638 AREX    -4029 ARGS      513 ARII     -576 ASEI    -2643 ASNA  
       293 ATNI     1476 ATSG    -4344 AVGR    -3224 AVID     1322 AX     -16357 AXAS      544 BABY  
      -323 BANF     -491 BBSI    -4900 BCLI     2131 BCOR    -5854 BCRX    -8615 BDSI     1131 BEAT  
     -2339 BGCP     2033 BGFV       70 BIIB      541 BJRI    -2518 BKCC     1538 BLBD    -1496 BLDR  
2016-08-01 12:59 positions:29 INFO .  
      -313 BLKB     1186 BLMN     -486 BNFT     -324 BOKF      966 BPMC     2315 BRCD    -1852 BRKL  
       823 BSET      530 BSTC      611 CA      -1792 CACQ     6754 CALA      589 CAR     -3503 CARA  
     -1921 CARB      398 CASH      457 CBSH      495 CDW     -2872 CECO     -825 CENT    -3053 CFMS  
      -825 CHRS    -4538 CLDX     6257 CLNE    -1451 CLVS     5274 CMRX    -1573 CMTL    -1822 CNCE  
     -1745 COLL     3036 CONN    -8672 COOP     6905 COWN      370 CPLA    -1421 CPTA     -769 CRAI  
       678 CRAY      609 CRMT    -1875 CROX      697 CSCO    -1267 CSFL     -495 CSOD     1275 CTRN  
       243 CTXS      952 CVTI    -1233 CYBE    35135 CYTR     3289 DAKT    -6291 DARE     1423 DFRG  
     -3714 DGLY     -841 DLTH     -593 DMRC     6762 DRNA    -1354 DVAX     -517 DWA      2530 DWSN  
     -1307 DXPE     2287 EHTH     1306 ELRC    -4266 EMKR    -2890 ENOC      966 ENTA     1001 EPAY  
     -1301 EPIQ    -2049 EPZM      -47 EQIX    -1993 ERII    -3180 ESIO    -1066 ESND    -1942 ESPR  
2016-08-01 12:59 positions:29 INFO .  
       632 EWBC     1480 EXA      2342 EXEL     -426 EXLS      182 EXPE     2391 EZPW      610 FARO  
     -2930 FCEL       83 FCNC_A  -1322 FDUS     -557 FELE    -1228 FEYE      970 FINL     -417 FIVE  
     -1682 FIVN    -2327 FLWS     1143 FNBC     -797 FNGN     1039 FNHC     1133 FNSR    -3177 FOLD  
       985 FONR    -2258 FORM      519 FORR     1644 FTFT    -4178 FTR       939 FWON_A  -2756 GAIN  
     -1103 GBDC     1192 GENC     -526 GIII      270 GILD    -2637 GLAD    -4691 GLBL    -1679 GLDC  
     -8816 GLUU    -5182 GNCA    -2574 GOGO     -883 GPRE    -1635 GPRO    -4422 GRPN      726 GTLS  
       469 HA       1307 HABT     -260 HAS      -600 HAWK    -3910 HBP     -1770 HDP     -4384 HDSN  
       611 HIBB    -6497 HLIT    -1149 HMHC     -935 HMST      553 HOLX     -564 HPT     -1107 HRTX  
      1102 HSII     1146 HTLD      523 HUBG      744 HWC       792 IBOC     -514 ICFI     1466 IESC  
       554 IIIN    -7410 IMGN      570 IMKT_A  -2855 IMMR     -418 INDB    12841 INFI    -1827 INSM  
2016-08-01 12:59 positions:29 INFO .  
      1367 INSY      743 INTL      192 INTU    -2430 INWK     -613 IPAR     -452 IPHS      688 IPXL  
      1055 IQNT      556 IRBT     2439 IRDM     1128 IRMD      634 ISCA      489 ITRI    -2019 IVTY  
      2294 JAKK     1164 JBLU     -433 JBSS      233 KALU    -1679 KLIC    -2981 KNDI    -2829 KPTI  
     -4825 KTOS    -5094 KURA     -263 LAMR    -2667 LAYN    -1487 LE      -1418 LFVN     -522 LGIH  
       478 LHCG    -4739 LIOX    -1233 LJPC    -1600 LOB       525 LOPE     -616 LORL    -5775 LPCN  
       805 LPLA      367 LPNT     2588 LQDT     3459 LSCC      897 LTRP_A  -3578 MACK     -638 MAT  
     -1238 MB        564 MBFI     2096 MCRB     1504 MDRX      337 MDVN     2934 MEET     -138 MELI  
      2436 MERC     -865 MESG     -719 MFRM      317 MGLN     -806 MIK      -604 MKTO     -129 MKTX  
       185 MLAB      656 MLHR     1892 MNTA    -1356 MRCC    13374 MRNS      990 MRTN     4599 MRTX  
      -543 MSCC      376 MSFT      123 MSTR     1374 MTGE     1112 MTRX      446 MTSC      687 MYGN  
2016-08-01 12:59 positions:29 INFO .  
      -377 NATH      665 NATL     -235 NFLX    -2122 NH        636 NHTC    -7177 NIHD     -724 NILE  
     -2045 NLNK     -799 NSIT     1590 NSTG      805 NTAP      411 NTGR     -222 NTK      1329 NUAN  
       342 NUVA       70 NWLI     1644 NWSA    -3663 OCLR     3853 OCSL    -4849 OCUL    -6013 ODP  
      -297 OLED     1739 OMED    -1330 OSTK     3177 OSUR    -1425 OTIC      403 OUTR    -4188 OVAS  
      3832 OXSQ     1690 PBPB      437 PDCO     6017 PDLI      965 PEBO    -2739 PEIX     -998 PERY  
       665 PFBC    -2165 PFSW    -4097 PGLC    -1074 PI        650 PINC     2213 PLAB      486 PLAY  
      1713 PLCM   -11937 PLUG      252 PLUS      462 PLXS     -207 POOL    -1187 PRIM    -3204 PRKR  
       437 PRSC    -1630 PRTK     2594 PSEC    -2724 PSTB     3468 PTCT     1367 QLGC    -1449 RAIL  
       928 RAMP     1965 RCII    -2717 RCKT    -1435 RDEN    -1429 RECN     1914 REGI   -28110 REXX  
     -5838 RGLS    -2596 RGNX    -8867 RIGL    -1992 RLOC     -665 RLYP      660 RMR      -654 RNST  
2016-08-01 12:59 positions:29 INFO .  
     -1508 RPD      2126 RPXC    -1191 RRD       440 RRGB      892 RRR     -4182 RUN       932 RUSH_A  
     -1568 RVNC      252 SAFM     2545 SALE      840 SANM      549 SATS      499 SAVE      416 SCAI  
       511 SCHL     2021 SCLN     1835 SCMP      518 SCSC     -619 SCTY    -1440 SCWX     1236 SEDG  
       573 SENE_A  -5371 SESN      430 SFBS     -401 SFLY     -456 SFNC     3964 SGI     -3235 SGMO  
     -1394 SHLD      663 SHLM    -2972 SIEN    -3200 SIGM      818 SIR      3477 SKUL     -425 SLGN  
     -2949 SLM     -1036 SLRC   -46581 SLS       894 SNBR      391 SNPS      563 SOHU      967 SPKE  
      1173 SPOK    -2103 SPWH    -1471 SPWR    -3265 SRNE     -832 SRPT    11120 SRRA      296 SSB  
       730 STLD      277 STMP      478 STRA    -7012 SUNW     1031 SYMC     6854 SYNC     1527 TAX  
     -4807 TBRA     -433 TCBI    -1328 TCPC     1838 TCRD      274 TECD     2011 TERP    -1154 TFSL  
     -3556 TGTX    -1197 TILE    -1122 TIVO     1032 TLMR      459 TMUS    -3337 TNDM      946 TOWN  
2016-08-01 12:59 positions:33 INFO .  
      1410 TPCO    -1388 TPIC     -805 TRMK     -299 TROW    -1222 TRS     -2250 TRUE    -1395 TRUP  
     -3319 TRVN      -91 TSLA    -3281 TTOO     5353 TTPH      525 TTWO    -1889 TUBE    -2615 TUES  
      1050 TYPE    -1599 UBNK    -1094 UCBI     2074 UEPS      518 UFCS      178 UFPI     1376 UIHC  
      4530 USAT     -867 UTEK      175 UTHR      381 VA      -1283 VECO     3009 VIAV    -1475 VRA  
       602 VRNT     -250 VRTS    -2603 VUZI    -1709 WATT      451 WDC       854 WERN      720 WFM  
      2351 WIFI      302 WOOF      487 WRLD    -8431 XCOM     -940 XELA    -2114 XONE      660 XPER  
     -1836 YRCW    -3348 ZAGG    -6968 ZAIS     6959 ZFGN     -457 ZG      -2318 ZGNX      778 ZION  
2016-08-01 13:00 WARN Your order for 113 shares of NWLI has been partially filled. 70 shares were successfully purchased. 43 shares were not filled by the end of day and were canceled.  
2016-08-01 13:00 WARN Your order for -1956 shares of WIBC failed to fill by the end of day and was canceled.  
2016-08-01 13:00 WARN Your order for -8525 shares of BCLI has been partially filled. 5050 shares were successfully sold. 3475 shares were not filled by the end of day and were canceled.  
2016-08-01 13:00 WARN Your order for -2156 shares of XELA has been partially filled. 940 shares were successfully sold. 1216 shares were not filled by the end of day and were canceled.  
2016-08-01 13:00 WARN Your order for -4614 shares of RLOC has been partially filled. 2172 shares were successfully sold. 2442 shares were not filled by the end of day and were canceled.  
2016-08-01 13:00 WARN Your order for -435 shares of NATH has been partially filled. 417 shares were successfully sold. 18 shares were not filled by the end of day and were canceled.  
2016-08-01 13:00 WARN Your order for -7308 shares of DARE has been partially filled. 6323 shares were successfully sold. 985 shares were not filled by the end of day and were canceled.
1 response
Blue Seahawk
Mar 28, 2019

A more generic function.

def lg(lines):  
    '''  
    (l)og (g)roups of output efficiently.  
    lines - A list of lines to log.  
    '''  
    if not lines:  
        return  
    buffer_len = 1024   # each group  
    chunk = ':'  
    for line in lines:  
        if line is None or not len(line):  
            continue  # skip if empty string for example  
        if len(chunk) + len(line) < buffer_len:  
            # Add to chunk if will still be under buffer_len  
            chunk += '\n{}'.format(line)  
        else:  # Or log chunk and start over with new line.  
            log.info(chunk)  
            chunk = ':\n{}'.format(line)  
    if len(chunk) > 2:       # if anything remaining  
        log.info(chunk)
About Quantopian
Careers
Community
Events
Documentation
Academia
Lectures
Status
Facebook
Twitter
LinkedIn
YouTube
Terms of Use
Privacy Policy

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by Quantopian.

In addition, the material offers no opinion with respect to the suitability of any security or specific investment. No information contained herein should be regarded as a suggestion to engage in or refrain from any investment-related course of action as none of Quantopian nor any of its affiliates is undertaking to provide investment advice, act as an adviser to any plan or entity subject to the Employee Retirement Income Security Act of 1974, as amended, individual retirement account or individual retirement annuity, or give advice in a fiduciary capacity with respect to the materials presented herein. If you are an individual retirement or other investor, contact your financial advisor or other fiduciary unrelated to Quantopian about whether any given investment idea, strategy, product or service described herein may be appropriate for your circumstances. All investments involve risk, including loss of principal. Quantopian makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances.