IRC logs for #openrisc Saturday, 2014-11-01

--- Log opened Sat Nov 01 00:00:03 2014
poke53281stekern: I am puzzled01:04
poke53281How does the ompic work?01:05
poke53281Finally I compiled a smp kernel.01:06
poke53281It boots, no problem. Just one core yet.01:06
poke53281But the ompic is never used01:06
poke53281Do I have to change the interrupt-parent in the dts file?01:07
poke53281good, smp is implemented. But I get a black screen :(06:26
poke53281no black screen anymore. But it seems, that the second is not started.06:39
stekernpoke53281: there's an example .dts here:
stekern(if you hadn't figured the dts problem out already)07:31
stekerntime to fix this atomic instruction emulation I think07:54
stekernhmm, looking at the MIPS emulation of ll/sc, I can't seem to see them actually emulating the link09:58
stekernah, no, now I see it09:59
mor1kx[mor1kx] bandvig pushed 1 new commit to withfpu:
mor1kxmor1kx/withfpu 1abd30b Andrey Bacherov: re-factoring floating point adder11:45
poke53281stekern: Yes, I read the dts files. But I don't know how ompic and the normal pic is related to each other.14:35
poke53281Anhow: I get the kernel message "Brought up 2 CPUs" now.  :)14:35
poke53281But then it hangs.14:36
poke53281And finally, he uses the ompic14:55
stekerncurrently, both are active and not related in any other way than that you have to connect the irq output of the ipi part of the ompic to a irq line of the pic, since the "pic parts" are not yet implemented in ompic15:17
stekernand you have to connect it to a line that's non-maskable, since otherwise you will not have it unmasked on all cpus15:18
poke53281obviously I don't have a clue how an interrupt controller works on smp machines.16:55
poke53281ompic_raise_softirq checks for 10ms whether an irq is pending on the destination CPU.17:00
poke53281then he sets the control register of the source cpu to 0x4001000017:02
poke53281irq=0 dst_cpu=1 and irq_gen17:03
poke53281then he sets the ctrl of cpu 1 to irq=0 dst_cpu=0 and irq_gen.17:03
poke53281so if I raise an interrupt (Exception int) how do I decide which cpu?17:11
poke53281are hardware (device) interrupts connected to the pic or to the ompic?17:14
poke53281Interrupt 1 is non-maskable by default. The same is true for interrupt 0.18:28
stekernperipheral interrupts are directly connected to the pic. so far, ompic only handles IPI19:58
poke53281and which core should be triggered if an interrupt is raised?19:59
poke53281dependent on the IEE Flag?19:59
stekernno, only the core that is assigned to the interrupt20:00
stekernthere's no support for migrating irq20:00
stekerns between cores (yet)20:00
poke53281but interrupt 1 is non-masked. That means, it is triggered for all cores?20:01
stekernyes, but you have num_cores irq lines coming out of the ipi part of ompic20:01
stekerneach core is connected to a seperate line20:02
stekernand the irq initiator cpu selects which other cpu should be interrupted20:02
poke53281sorry, for slow physicists please. Let's say my uart triggers irq no. 2. Is the ompic involved in this irq?20:04
poke53281each core has a separate line. This I understand. And how do I figure out, which core is responsible for which irq? Does the PICMR has anything to do with it?20:06
poke53281Each core has a separate PIC MR and SR register?20:07
poke53281At the moment I have the following: A pic interrupt sets the SR registers of all cores. Clear clears all SR registers.20:15
poke53281Of course only the core with the correct mask in the MR register will handle the interrupt in the end.20:15
poke53281But that would mean, that the ompic is the only one which can trigger single cores? But then it shouldn't trigger irq 1 of the pic.20:19
stekernpoke53281: ok, let me try to start over ;)20:53
stekern1) ompic currently have one function, to implement ipi (inter processor interrupts). Long term, I want to add more functionality to it, and not use the builtin PIC.20:55
stekern2) due to 1), for a uart interrupt, ompic is not involved. It's just a 'normal' peripheral interrupt.20:56
stekernas you say, the ipi part of ompic is the only thing that can trigger irqs on a single cpu (if we disregard the masking)20:58
stekernand the ipi interrupts are different from the normal peripheral interrupts, each ipi interrupt has an own signal for each cpu, while the peripherals connect the same signal to all the cpus20:59
stekernmight make it more clear21:01
poke53281so if the ompic non-maskable interrupt is triggered (int 1) it should only trigger the core (set PIC SR) that is defined by the ompic. All other interrupts should set the PIC  SR register of all cores?21:02
poke53281and clear should behave the same.21:04
poke53281Ok, so the next question is then, how the ompic works.21:04
stekernthe code is here:
poke53281Read STAT of CPU 021:05
poke53281Write CTRL of CPU 1 :  dstcpu=0 irqno=0 flags=1 (GEN)21:05
stekernit's basically just an array of registers21:06
poke53281Hmm, I can try to undestand it.21:06
poke53281So If I write the the control of cpu 1 does that mean that I trigger the line of cpu 1 or do I trigger the line of dstcpu?21:07
stekernyou trigger the line of dstcpu21:08
stekernif you write the control of cpu1, that means you should be cpu121:08
stekernthere's really nothing enforcing it, but that's how it's supposed to be used21:09
poke53281Ahh, Ok. In principle I can use any control register. At least for my emulation.21:09
poke53281Because the data is no longer used in the end.21:10
stekernhmm, how do you mean?21:10
poke53281It just tells me, which cpu triggers the soft IRQ. But this information is never used.21:10
stekernah, you might be right. I just figured it might be interesting to know the source cpu when I designed it-21:11
poke53281Ok, finally I understand :)21:13
poke53281In principle one control register might be sufficient.21:13
stekernbut you still need to have a control register per cpu21:13
poke53281Well, two cpus could write at the same time.21:14
poke53281But we have a lock there.21:14
stekernsince several cpus might want to send at the same time21:14
stekernbut the lock is only there to protect against re-entrance from the same core21:14
poke53281Ahh, Ok.21:15
poke53281But if we would use a global lock, then one CTRL register would be sufficient.21:15
stekernthen you would need to lock until the other cpu have handled the irq21:16
stekern...which I don't think will work21:17
poke53281well, this delay with 10000 retries is a little bit weird.21:17
stekernit only does it if we're still waiting for a retreiver to handle an earlier ipi request21:18
poke53281I can work this way.21:18
poke53281And what happens if there are three requests.21:19
stekernthere can't be three requests to the same destination21:19
poke53281One is acknoledged immediately, so the second core can send the IRQ_GEN right in time. 10ms should be more than enough.21:20
poke53281Two is the limit?21:20
stekernnumber two will be waiting in the retry loop for the first to be handled, the third will be spinning in the spinlock waiting for the lock to open21:21
stekernhow do you know that the first is acknowledged immediately?21:22
stekern(I assume you meant cleared from the receiver side by that)21:23
poke53281well, under the assumption, that the core is nothing else to do.21:23
stekernit might have interrupts turned off ;)21:23
poke53281Ok, I think you are right. Those are tiny details which would fail horribly if done wrong.21:24
stekernIIRC I think I saw some time outs when I had it set at 1000, so I raised it to 10 00021:25
poke53281There must be a better way for the timeouts.21:27
poke53281ompic_ipi_lock? Is this a global variable. Or several global variables for each core?21:28
poke53281Is ompic_ipi_lock one global variable that is the same for each core?21:30
poke53281and _irqsave means that the interrupts are enabled during the waiting time in the spinlock?21:31
stekernwell, the loop could be an infinite loop, the timeout is just there to make a failure evident21:32
stekernirqsave means interrupts are *disabled*21:35
stekernthey need to be disabled, because ompic_raise_softirq() can be called from an interrupt context21:37
stekernwhich would mean that you'd have a deadlock on one core trying to obtain a lock that it's holding21:37
stekernlocks are (of course) not per-cpu, it's a global 'variable'21:38
stekernthis is the actual implementation:
stekernso the actual lock works by splitting up a 32-bit variable in two 16-bit fields, one next field and one owner field21:40
stekernticket spinlocks works like a ticket system when you're queuing at a bank21:42
stekernso at the beginning, there's a number 0 at the ticket machine and a number 0 at the display21:44
stekernthe first one to arrive takes ticket 0 and get served. and at the ticket machine there'll be ticket 1 waiting for the next to arrive21:44
poke53281I see21:45
stekernwhen the first cpu is done, it unlocks the lock by setting the 'display' to 121:45
stekernthe benefit is of course when you have several cores waiting for the lock, they then get served in the order they arrived21:48
poke53281That makes sense.21:49
poke53281Ok, I try to implement the ompic now.21:49
poke5328118 more kernel messages before it stops.21:58
poke53281Ok, next problen: Timer :)22:03
stekernnice ;)22:14
poke53281At the moment the kernel just stops at random positions. But my TTCRs are most of the time out of sync.22:16
poke53281TTCRs must always be perfect synchronously.22:17
poke53281So, is it also true with TTMRs?22:18
poke53281So, the 100Hz tick is synchronous to all cores?22:18
stekernthe capture interrupts are set independently on each core, but the timer reference has to be in sync22:20
stekernthe tick timer is used as two different 'devices' - a clock_envent_device and a clocksource22:23
poke53281But I should get at least a shell with out of sync timers?22:23
poke53281Because, this is the only thing, that is wrong currently.22:24
stekernnah, if the timers are out of sync, it might think that the time reference is something (that it got from another core), then setup the next event based on this reference22:24
stekernso, the correct way to fix this is to have a globally accessible timer with per-cpu capture interrupts22:27
poke53281yes, and this timer is currently always zero in my implementation. :)22:27
poke53281this global timer I mean.22:28
stekernbut currently, the kernel code kind of expects that the built-in timer is there, so it needs some work22:28
stekernthe openrisc kernel code22:28
stekernI'm tired and can't think straight how C sign extension and integer promotion works, will this sign extend properly?22:30
stekernlong imm; imm = (short)insn;22:31
stekernwhere the imm is in [15:0] of insn22:31
poke53281I think so22:33
stekernyeah, I think that's right too22:38
poke53281timer in sync now22:53
poke53281one kernel message before the shell I get a OMPIC timed out message.22:56
poke53281or before he starts init I mean.22:56
poke53281So close22:58
stekernit panics here22:59
stekerncan I scroll up in the console somehow?22:59
poke53281Yes, after a while22:59
stekerngreat ;)22:59
poke53281I know this problem :)22:59
poke53281try it again. My solution to the problem23:03
poke53281self-detected stall on CPU 023:03
poke53281I know your next question.23:06
stekernwell, that is because the ompic failed23:06
stekerndo a quick test and increase that timeout (or remove it completely)23:07
poke53281your next question is, what the pc of cpu 0 is and in what function is fails :)23:07
stekernI think I have rudimentary l.lwa/l.swa emulation now23:09
stekernlet's see if it works23:10
poke53281? for smp ?23:10
stekernI'm not really breaking the link on context switch yet, but it should at least do something23:10
stekernno, for up23:10
stekernjust so you can run userspace code that contains l.lwa/l.swa without crashing on implementations that lack them23:11
poke53281Ok, I understand. You mean an unknown instruction exception is triggered and lwa and swa is emulated in the kernel.23:13
stekernnow I just have the problem that I don't think I have an or1ksim without them23:13
stekernyes, exactly23:13
stekernoh, right... calculating the return pc is going to be mighty fun...23:23
stekernlet's pretend that you can't put l.lwa and l.swa in delay slots to begin with, then it's easy23:30
poke53281Let's pretend, that removing the dealay slots in all (important) OpenRISC related code would take less than one day.23:32
poke53281Increased the number of retries by a factor of 100. No luck.23:34
stekernwell, the biggest issue, the toolchain is mostly covered23:36
stekernkernel and then libc shouldn't be insane amount of work23:36
stekernwill take more than a day for sure ;)23:37
stekernwhen I have a delay-slot-less cappuccino I'll probably do that23:38
poke53281Oh damn. Found the error. Stupid poke23:40
stekernlooks like the emulation works now23:46
stekernemulation of l.lwa and l.swa in the kernel doesn't seem to play well though23:54
poke53281Well, that's important :)23:55
stekernbut maybe that's because I don't actually clear the flag ever23:55
poke53281Only a tiny little bit is missing23:55
stekernwhat's that?23:56
poke53281Sometimes the init routine runs, but bash is not starting.23:56
poke53281So, he gets even an IP from the relay.23:57
stekernah, it started here ;)23:57
stekerndon't you implement the 'new' version registers?23:57
stekernalso, are you actually emulating caches?23:58
stekernyou could save a couple of milliseconds in the boot by claiming not to have them23:59
poke53281I didn't touch the version or upr register since the first version.23:59
poke53281of jor1k23:59
stekernbut, way cool progress so far!23:59
--- Log closed Sun Nov 02 00:00:05 2014

Generated by 2.15.2 by Marius Gedminas - find it at!