Skip to content

Fix CUDA Extension Compilation and CPU Fallback Logic #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Electroiscoding wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Fix CUDA Extension Compilation and CPU Fallback Logic #6

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 81 additions & 0 deletions benchmark_results/20260317_053035/benchmark_results.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
impl,case,status,cold_load_time_ms,warm_load_time_ms,tokens_produced,bytes_processed,avg_time_ms,tokens_per_sec,mb_per_sec,notes
crayon:cpu:lite,english,OK,14.868305000049986,9.87663299997621,144001,740000,8.647791299995333,16651766.330216331,81.60684846094816,
crayon:cpu:lite,code,OK,9.026592999930472,8.920501999909902,556000,1048000,19.12290340000027,29075082.814045496,52.26458883820623,
crayon:cpu:lite,unicode,OK,12.342592999971203,7.844900999998572,474000,660000,12.112038599980224,39134617.68531467,51.966895880694494,
crayon:cpu:lite,mixed,OK,12.142413999981727,8.46946800004389,640000,1397500,26.57187120001936,24085620.285542168,50.156793518431755,
crayon:cpu:standard,english,OK,56.67616999994607,56.14077899997483,136001,740000,8.373979399993914,16240904.533404853,84.2752245295872,
crayon:cpu:standard,code,OK,59.37171699997634,60.828852999975425,312000,1048000,19.83747910001057,15727804.849953631,50.38194009203607,
crayon:cpu:standard,unicode,OK,55.76143799999045,60.44815200004905,216001,660000,8.230683800013594,26243384.541104987,76.47299594076077,
crayon:cpu:standard,mixed,OK,53.84370900003432,59.3777520000458,372501,1397500,24.60571969999137,15138797.179752098,54.16463624830295,
crayon:cpu:lite,english,OK,12.34249699996326,8.38803600004212,144001,740000,8.31000389998735,17328632.059994485,84.92402682767685,
crayon:cpu:lite,code,OK,8.134915000027831,8.08777400004601,556000,1048000,18.63192479999043,29841253.97502064,53.64183756228253,
crayon:cpu:lite,unicode,OK,8.606406000012612,8.018338999931984,474000,660000,11.482100199975775,41281646.36648964,54.81793729943699,
crayon:cpu:lite,mixed,OK,8.287248999977237,8.105816000011146,640000,1397500,25.149739800031057,25447579.382082105,52.99298791059813,
crayon:cpu:standard,english,OK,57.03588000005766,55.99314200003391,136001,740000,8.245439300003454,16494087.8286428,85.5890109021031,
crayon:cpu:standard,code,OK,52.5738100000126,55.370928000002095,312000,1048000,19.90267890000723,15676281.648692358,50.216892339723515,
crayon:cpu:standard,unicode,OK,51.06038899998566,56.10423499990702,216001,660000,8.22087299998202,26274703.42875658,76.56425890893844,
crayon:cpu:standard,mixed,OK,50.55269899992254,56.21722000000773,372501,1397500,24.131329999977424,15436405.701647962,55.229440614296074,
crayon:cpu:lite,english,OK,12.330100999975002,8.000930999969569,144001,740000,10.23242560000881,14073007.283813138,68.968885944308,
crayon:cpu:lite,code,OK,8.94056599997839,7.907703000000765,556000,1048000,40.71248950000381,13656742.852827704,24.548994568206307,
crayon:cpu:lite,unicode,OK,11.877349999963371,8.837169999992511,474000,660000,13.601785399976052,34848366.303502664,46.27517861215728,
crayon:cpu:lite,mixed,OK,10.845587999938289,9.425191999980598,640000,1397500,29.756448500006627,21507943.05979954,44.788942375883245,
crayon:cpu:standard,english,OK,68.05869800007258,71.99046100004125,136001,740000,9.61803189999273,14140210.951068152,73.37457407904402,
crayon:cpu:standard,code,OK,61.89855200000238,63.58751300001586,312000,1048000,22.32071649999625,13978045.91084934,44.7768190413564,
crayon:cpu:standard,unicode,OK,63.469779000001836,61.042190000080154,216001,660000,10.250069699998221,21073124.995436616,61.406904269951866,
crayon:cpu:standard,mixed,OK,78.44032599996353,67.1652720000111,372501,1397500,29.630535900025734,12571524.229491796,44.979269415663076,
crayon:cpu:lite,english,OK,14.983427999936794,9.01762300009068,144001,740000,8.486741399985931,16967761.030192193,83.15547285814493,
crayon:cpu:lite,code,OK,10.302792999937083,12.465697000038745,556000,1048000,21.7420526000069,25572562.54636343,45.968552370875635,
crayon:cpu:lite,unicode,OK,8.5818789999621,10.996005000038167,474000,660000,14.206197699979839,33365718.963679682,44.30636980569532,
crayon:cpu:lite,mixed,OK,8.1939039999952,7.966666000015721,640000,1397500,31.072842300000048,20596764.010867424,42.8914691585112,
crayon:cpu:standard,english,OK,75.51630000000387,69.93360599994958,136001,740000,10.2323704000014,13291250.676381044,68.96925800697446,
crayon:cpu:standard,code,OK,68.99347599994599,75.94491300005757,312000,1048000,23.726231499995265,13150002.350776281,42.124291149816585,
crayon:cpu:standard,unicode,OK,62.730942000030154,73.37964800001373,216001,660000,11.02782569998908,19586907.326637734,57.07607881658379,
crayon:cpu:standard,mixed,OK,62.92240700008733,67.83681499996419,372501,1397500,25.266263900005015,14743018.65421132,52.74859244929638,
crayon:cpu:lite,english,OK,14.862322999988464,8.51527999998325,144001,740000,9.711931699996512,14827225.360331943,72.66515209748421,
crayon:cpu:lite,code,OK,12.431329000037294,9.098590999997214,556000,1048000,20.486216000006152,27140200.02521857,48.78649544617951,
crayon:cpu:lite,unicode,OK,10.318500999915159,9.489746999975068,474000,660000,13.82918060000975,34275349.61830391,45.51426921329534,
crayon:cpu:lite,mixed,OK,8.992912999929104,8.828263999930641,640000,1397500,30.61809850001964,20902669.706924792,43.528498583178816,
crayon:cpu:standard,english,OK,63.77301999998508,68.11127399998895,136001,740000,9.099823600001855,14945454.54704993,77.55304115350995,
crayon:cpu:standard,code,OK,70.96236099994258,65.23993500002234,312000,1048000,21.603614000014204,14442028.079181328,46.26312447505741,
crayon:cpu:standard,unicode,OK,64.17608200001723,68.946622999988,216001,660000,9.849700700010544,21929701.884217534,63.902961927305185,
crayon:cpu:standard,mixed,OK,61.973479000016596,66.58864000007725,372501,1397500,26.984551500004272,13804231.654542822,49.38973535200403,
crayon:cpu:lite,english,OK,14.038064000033046,8.293565999906605,144001,740000,9.331666100001712,15431435.1217489,75.62625865283538,
crayon:cpu:lite,code,OK,9.214194000037423,11.521090999963235,556000,1048000,20.19936870000265,27525612.718774077,49.47930296423664,
crayon:cpu:lite,unicode,OK,10.631344999978865,9.76838700000826,474000,660000,14.771757600021829,32088260.099752758,42.61003097066762,
crayon:cpu:lite,mixed,OK,11.537464999946678,10.287465999908818,640000,1397500,30.200901099999555,21191420.675855577,44.129804364604006,
crayon:cpu:standard,english,OK,70.90438300008373,68.66426600004161,136001,740000,8.855051299974548,15358578.442158874,79.69677082984866,
crayon:cpu:standard,code,OK,67.25170300001082,65.2393130000064,312000,1048000,20.73662330002435,15045844.035737181,48.197368931931,
crayon:cpu:standard,unicode,OK,67.00485500005016,68.079785000009,216001,660000,8.824563100017713,24477245.791303415,71.32648287447358,
crayon:cpu:standard,mixed,OK,58.914462999950956,68.05693100000099,372501,1397500,26.207790499984185,14213369.112524949,50.853575664020156,
crayon:cpu:lite,english,OK,12.835383000037837,9.33742900008383,144001,740000,8.70022369997514,16551413.499909371,81.11504008139946,
crayon:cpu:lite,code,OK,9.479603000045245,8.138844999962203,556000,1048000,21.23308629999201,26185547.976612765,47.07043853507656,
crayon:cpu:lite,unicode,OK,8.614314000055856,9.598793999998634,474000,660000,15.694109399998979,30202414.671585687,40.10581504090738,
crayon:cpu:lite,mixed,OK,11.087617000043792,8.254261000047336,640000,1397500,29.928679299996475,21384171.135145124,44.53119510615663,
crayon:cpu:standard,english,OK,74.17762199997924,73.78222199997708,136001,740000,8.760950500015952,15523543.935073296,80.55278866594898,
crayon:cpu:standard,code,OK,66.53267000001506,75.45856199999434,312000,1048000,20.645504200001596,15112248.99026568,48.41008841012818,
crayon:cpu:standard,unicode,OK,65.45338000000811,88.04373200007376,216001,660000,8.822801700000582,24482132.472725276,71.34072262194032,
crayon:cpu:standard,mixed,OK,72.09946499995112,70.31428500010861,372501,1397500,27.636978300017745,13478354.831568573,48.22379070207102,
crayon:cpu:lite,english,OK,21.455864000017755,9.034896000002846,144001,740000,10.249723800006905,14049256.624837343,68.85248889732517,
crayon:cpu:lite,code,OK,11.17282100005923,11.382386999912342,556000,1048000,21.1772855999925,26254545.105638888,47.19446592315419,
crayon:cpu:lite,unicode,OK,8.655612999973528,10.659256999929312,474000,660000,15.423255300004257,30732811.639308672,40.81012967657686,
crayon:cpu:lite,mixed,OK,10.22083299994847,9.119001000044591,640000,1397500,28.74799730000177,22262420.3460587,46.360094001318565,
crayon:cpu:standard,english,OK,58.88794100007999,56.890226000064104,136001,740000,7.4119927999959145,18348776.59353298,95.21312461893028,
crayon:cpu:standard,code,OK,54.90327600000455,55.97901500004809,312000,1048000,17.434602800005905,17895446.4050018,57.32569276504604,
crayon:cpu:standard,unicode,OK,50.7556890000842,55.61820000002626,216001,660000,7.618974400008938,28350403.697346274,82.61283156790586,
crayon:cpu:standard,mixed,OK,50.34348299989233,55.09503400003268,372501,1397500,21.914746300012666,16997732.709312025,60.815664435821645,
crayon:cpu:lite,english,OK,12.457188999974278,8.017877000042972,144001,740000,7.411144900015643,19430331.202901736,95.22401783550818,
crayon:cpu:lite,code,OK,8.076985000002423,7.703710999976465,556000,1048000,16.831052300005922,33034179.330534454,59.381354521333066,
crayon:cpu:lite,unicode,OK,8.161683000025732,7.643239999993057,474000,660000,11.583916900008262,40918801.825974934,54.33611569051191,
crayon:cpu:lite,mixed,OK,8.703353000100833,8.032742999944276,640000,1397500,24.614256099994236,26001192.049031693,54.14585156518488,
crayon:cpu:standard,english,OK,55.18846500001473,55.58668399999078,136001,740000,7.250363000014204,18757819.436038382,97.33567741908128,
crayon:cpu:standard,code,OK,52.35724600004232,54.53782400002183,312000,1048000,17.24318060001906,18094109.621496115,57.9620840712325,
crayon:cpu:standard,unicode,OK,50.46295099998588,55.4346459999806,216001,660000,7.501434900018467,28794624.34573261,83.90728670145168,
crayon:cpu:standard,mixed,OK,49.663459000043986,54.793023000001995,372501,1397500,21.751640299999053,17125191.243623875,61.27169440080307,
crayon:cpu:lite,english,OK,12.217731999953685,8.143410000002405,144001,740000,7.4156978999667444,19418401.604607675,95.16555335186858,
crayon:cpu:lite,code,OK,8.170772999960718,7.94540800006871,556000,1048000,16.79275569999845,33109515.194105472,59.516776248572604,
crayon:cpu:lite,unicode,OK,8.131237000043257,7.8823059999422185,474000,660000,11.37300549999054,41677637.45479541,55.343774240560116,
crayon:cpu:lite,mixed,OK,8.195090999947752,7.952807000037865,640000,1397500,23.71097740000323,26991717.346915983,56.20855836915238,
crayon:cpu:standard,english,OK,55.76155500000368,55.5743900000607,136001,740000,7.294867399991745,18643382.057932116,96.74185361360011,
crayon:cpu:standard,code,OK,52.74450500007788,54.82426399998985,312000,1048000,17.348223099997995,17984550.82123287,57.61112696284529,
crayon:cpu:standard,unicode,OK,50.57152599999881,55.03631899989614,216001,660000,7.481271000028755,28872233.073654167,84.13343786446256,
crayon:cpu:standard,mixed,OK,49.62355600002866,55.199267000034524,372501,1397500,21.733554900015406,17139441.831475805,61.3226811402487,
Loading
Loading