using L2 inclusive-cache with non-default MemoryDataBits ?

[gsomlo]

I'm using (64-bit) RocketChip as part of the LiteX SoC environment, which provides DRAM controllers of varying AXI port width, depending on the specifics of individual FPGA boards it's being built for (64, 128, 256, or 512 bits).

I'm using the following in rocket-chip/src/main/scala/subsystem/Configs.scala to set the width for the MEM AXI port, to configure address routing for it and for the MMIO port, and to ensure that the frontend slave port has the same width as MMIO (MMIO and the frontend slave are normally kept at the default 64-bit AXI width):

        class WithMemoryDataBits(dataBits: Int) extends Config((site, here, up) => {
          case MemoryBusKey => up(MemoryBusKey, site).copy(beatBytes = dataBits/8)
        })

        class WithLitexMemPort extends Config((site, here, up) => {
          case ExtMem => Some(MemoryPortParams(MasterPortParams(
                              base = x"8000_0000",
                              size = x"8000_0000",
                              beatBytes = site(MemoryBusKey).beatBytes,
                              idBits = 4), 1))
        })

        class WithLitexMMIOPort extends Config((site, here, up) => {
          case ExtBus => Some(MasterPortParams(
                              base = x"1000_0000",
                              size = x"7000_0000",
                              beatBytes = site(SystemBusKey).beatBytes,
                              idBits = 4))
        })

        class WithLitexSlavePort extends Config((site, here, up) => {
          case ExtIn  => Some(SlavePortParams(
                              beatBytes = site(SystemBusKey).beatBytes,
                              idBits = 8,
                              sourceBits = 4))
        })

I then configure my RocketChip(s) like so:

class BaseLitexConfig extends Config(
  new WithLitexMemPort() ++
  new WithLitexMMIOPort() ++
  new WithLitexSlavePort ++
  new WithNExtTopInterrupts(8) ++
  new WithCoherentBusTopology ++
  new BaseConfig
)

class LitexConfig_linux_2_1 extends Config(
  new WithNBigCores(2) ++
  new WithMemoryDataBits(64) ++
  new WithInclusiveCache() ++
  new BaseLitexConfig
)

class LitexConfig_linux_2_2 extends Config(
  new WithNBigCores(2) ++
  new WithMemoryDataBits(128) ++
  new WithInclusiveCache() ++
  new BaseLitexConfig
)

Long story short, the variant configured using WithMemoryDataBits(64) works fine (with an external axi width adapter if needed for DRAM controllers that are wider), whereas WithMemoryDataBits(128) locks up hard whenever attempting any memory accesses (on a board where the DRAM controller has a native 128-bit wide AXI port).

I'm wondering if it's reasonable to assume that the InclusiveCache doesn't expect WithMemoryDataBits() to differ from the default (64 bits for the 64-bit rocket chip), and that the internal-to-rocket width adapter somehow doesn't "fit" the added L2 cache?

I'm not super fluent in Chisel, so not ready to debug it myself, but wanted to start by asking if, as a first pass, this hypothesis makes sense.

Thanks much!

EDIT: I'm wondering if this line might explain the behavior I'm seeing, and if it should somehow be a parameter rather than a hard-coded value? https://github.com/chipsalliance/rocket-chip-inclusive-cache/blob/main/design/craft/inclusivecache/src/Configs.scala#L59

[gsomlo]

Also, if anyone with a clue can explain what the restrictions (or rather built-in expectations) of the relative widths of the various AXI ports (MEM vs. MMIO vs. frontend-slave (dma) are, that'd be super informative.

IOW, it appears I can set them independently of each other, but is that a good idea? (obviously "not if I want to add an L2 cache " :), but how about in principle?)