icache: Fix icache invalidation

This fixes two bugs in the flash invalidation of the icache.

The first is that an instruction could get executed twice.  The
i-cache RAM is 2 instructions (64 bits) wide, so one read can supply
results for 2 cycles.  The fetch1 stage tells icache when the address
is equal to the address of the previous cycle plus 4, and in cases
where that is true, bit 2 of the address is 1, and the previous cycle
was a cache hit, we just use the second word of the doubleword read
from the cache RAM.  However, the cache hit/miss logic also continues
to operate, so in the case where the first word hits but the second
word misses (because of an icache invalidation or a snoop occurring in
the first cycle), we supply the instruction from the data previously
read from the icache RAM but also stall fetch1 and start a cache
reload sequence, and subsequently supply the second instruction
again.  This fixes the issue by inhibiting req_is_miss and stall_out
when use_previous is true.

The second bug is that if an icache invalidation occurs while
reloading a line, we continue to reload the line, and make it valid
when the reload finishes, even though some of the data may have been
read before the invalidation occurred.  This adds a new state
STOP_RELOAD which we go to if an invalidation happens while we are in
CLR_TAG or WAIT_ACK state.  In STOP_RELOAD state we don't request any
more reads from memory and wait for the reads we have previously
requested to be acked, and then go to IDLE state.  Data returned is
still written to the icache RAM, but that doesn't matter because the
line is invalid and is never made valid.

Note that we don't have to worry about invalidations due to snooped
writes while reloading a line, because the wishbone arbiter won't
switch to another master once it has started sending our reload
requests to memory.  Thus a store to memory will either happen before
any of our reads have got to memory, or after we have finished the
reload (in which case we will no longer be in WAIT_ACK state).

Signed-off-by: Paul Mackerras <paulus@ozlabs.org>

icache.vhdl

@@ -171,7 +171,7 @@ architecture rtl of icache is
     signal eaa_priv  : std_ulogic;
 
     -- Cache reload state machine
-    type state_t is (IDLE, CLR_TAG, WAIT_ACK);
+    type state_t is (IDLE, STOP_RELOAD, CLR_TAG, WAIT_ACK);
 
     type reg_internal_t is record
 	-- Cache hit state (Latches for 1 cycle BRAM access)
@@ -546,7 +546,7 @@ begin
 	end loop;
 
 	-- Generate the "hit" and "miss" signals for the synchronous blocks
-        if i_in.req = '1' and access_ok = '1' and flush_in = '0' and rst = '0' then
+        if i_in.req = '1' and access_ok = '1' and flush_in = '0' and rst = '0' and use_previous = '0' then
             req_is_hit  <= is_hit;
             req_is_miss <= not is_hit;
         else
@@ -580,7 +580,7 @@ begin
         i_out.next_pred_ntaken <= i_in.pred_ntaken;
 
 	-- Stall fetch1 if we have a miss on cache or TLB or a protection fault
-	stall_out <= not (is_hit and access_ok);
+	stall_out <= not (is_hit and access_ok) and not use_previous;
 
 	-- Wishbone requests output (from the cache miss reload machine)
 	wishbone_out <= r.wb;
@@ -757,9 +757,15 @@ begin
 			r.wb.adr <= next_row_addr(r.wb.adr);
 		    end if;
 
+                    -- Abort reload if we get an invalidation
+                    if inval_in = '1' then
+                        r.wb.stb <= '0';
+                        r.state <= STOP_RELOAD;
+                    end if;
+
 		    -- Incoming acks processing
 		    if wishbone_in.ack = '1' then
-                        r.rows_valid(r.store_row mod ROW_PER_LINE) <= '1';
+                        r.rows_valid(r.store_row mod ROW_PER_LINE) <= not inval_in;
 			-- Check for completion
 			if is_last_row(r.store_row, r.end_row_ix) then
 			    -- Complete wishbone cycle
@@ -775,6 +781,18 @@ begin
 			-- Increment store row counter
 			r.store_row <= next_row(r.store_row);
 		    end if;
+
+                when STOP_RELOAD =>
+                    -- Wait for all outstanding requests to be satisfied, then
+                    -- go to IDLE state.
+                    if get_row_of_line(r.store_row) = get_row_of_line(get_row(r.wb.adr)) then
+                        r.wb.cyc <= '0';
+                        r.state <= IDLE;
+                    end if;
+                    if wishbone_in.ack = '1' then
+			-- Increment store row counter
+			r.store_row <= next_row(r.store_row);
+		    end if;
 		end case;
 	    end if;