.. _usage_run: Run ELEQTRONeX ========= Run --- Run locally ^^^^^^^^^^^ Upon compilation, executable is generated in ``Exec/`` folder in the ELEQTRONeX folder. To run the executable with an input file from the `input/` folder in the `ELEQTRONeX` folder, we can do the following: Assuming we are in the ``Exec/`` folder, to run with a single processor, execute: .. code-block:: bash ./ ../input// Ensure the `plot.folder_name` parameter in the input file specifies the output directory. Alternatively, you can specify this parameter on the command line in , which will override what is specified in the input file. For multiple processors, execute: .. code-block:: bash mpiexec -np ./ ../input// Run on Perlmutter ^^^^^^^^^^^^^^^^^^ To run the code on `Perlmutter `_, we can submit a jobscript as shown below for 2 nodes and 8 GPUs. .. code-block:: bash #!/bin/bash #SBATCH -N 2 #SBATCH -C gpu #SBATCH -G 8 #SBATCH -J #SBATCH -q regular #SBATCH --mail-user=saurabhsawant@lbl.gov #SBATCH --mail-type=ALL #SBATCH -t 00:30:00 #SBATCH -A m4540_g #SBATCH -o .o%j #SBATCH -e .e%j #OpenMP settings: export OMP_NUM_THREADS=1 export OMP_PLACES=threads export OMP_PROC_BIND=spread #run the application: srun -n 8 -c 32 --cpu_bind=cores -G 8 --gpu-bind=single:1 / / For more information on, see ``https://docs.nersc.gov/systems/perlmutter/running-jobs/``. NERSC also provides a script generator, ``https://my.nersc.gov/script_generator.php``. Inputs ------ Module enabling parameters ^^^^^^^^^^^^^^^^^^^^^^^^^^ The following flags enable particular modules. - ``use_electrostatic = 1`` Electrostatics module enabled. - ``use_transport = 1`` Transport module enabled. - ``transport.use_negf = 1`` NEGF solver enabled. - ``use_diagnostics = 1`` Diagnostics are enabled. - ``domain.embedded_boundary = 1`` Set embedded boundaries in the domain. Field data output parameters ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - ``plot.folder_name = `` Replace ```` with the foldername. - ``plot.fields_to_plot = . vecField`` vector of ``multiFabs`` to be outputted. Replace with the name of multifab defined in ``macroscopic.fields_to_define``. If ```` is not specified, then data is outputted only in the output folder. ``vecField`` outputs the gradient of electrostatic potential in all three directions. Additionally, there is an option to create raw_fields folder as ``/raw_fields``, where raw data may be outputted. Replace ```` with 1 to output the multifab in the output folder as well as separately in the ``raw_fields`` folder along with the ghost cells. This may be useful while debugging. Replace ```` with 2 to output data only in the ``raw_fields`` folder. - ``plot.write_after_init = 1`` Data can be written out after initialization and before the first iteration. This may be useful for debugging before running the simulation. - ``plot.write_interval = `` If the previous parameter is enabled, then this parameter can be used to specify the interval with which the data is written out. - ``plot.rawfield_write_interval = `` This interval can be different from ``plot.write_interval``. Boundary Conditions ^^^^^^^^^^^^^^^^^^^ - ``domain.is_periodic = 0 1 0`` sets whether domain is periodic in the X, Y, Z directions, respectively. In this example, `Y` direction is periodic. - ``boundary.hi = () () ()`` This parameter sets the maximum domain boundaries in the `X`, `Y`, and `Z` directions. Replace ``<*_boundary_type>`` by ``dir`` (Dirichlet), ``rob`` (Robin), ``neu`` (Neumann), or ``per`` (Periodic). We can specify floating point values on Dirichlet and Neumann boundaries for electrostatic potential in paranthesis. If no value is specified then ``0.0`` is assumed. We may choose to specify a string parameter in the paranthesis such as ``dir(Zmax)`` to specify a time and/or spatially varying function. In this case, we need to specify another parameter, ``boundary._function``, for parsing the function, such as ``boundary.Zmax_function = "10*cos(2*pi*x/(2*Lx))"``. See Parser in the AMReX documentation. For Robin boundaries, we need to set a string parameter, for example ``rob(Ymax)`` and set three more Robin boundary specific parameters as ``boundary._a_function``, ``boundary._b_function``, ``boundary._f_function``. If ``domain.is_periodic`` is specified to be periodic, then it overrides the ``boundary.hi`` and ``boundary.lo`` parameters. - ``boundary.lo = neu(-0.5) neu dir(5)`` Similarly, set the minimum domain boundaries in the `X`, `Y`, and `Z` directions. In this example, minimum `X`, `Y` boundaries are Neumann with values of -0.5 and 0., while minimum `Z` boundary is Dirichlet with a potential value of 5~V. Embedded boundary parameters ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ For these parameters to work, we enable ``domain.embedded_boundary=1``. - ``domain.specify_using_eb2=`` If this is true, then we load EBs provided by AMReX using its inbult interface. Typically set to 0. - ``ebgeom.objects= ...`` Here we can set EB object labels. If we want to set more than 1 labels, we can do that by listing space-separated labels. For example, we can define labels as Source, Drain. Gate can be specified using either EB or the boundary. If we are setting gate using EB, we should use the label ``Gate``. - ``ebgeom.specify_inhomo_dir=1`` This means we specify inhomogeneous Dirichlet boundaries on the EB. We can set different Dirichlet boundaries on each EB. - ``.geom_type = `` can be ``box`` to set a box-shaped EB. Or it can be ``cntfet_contact_cyl`` to define a cylindrical EB shape with a cylindrical cavity (in which carbon nanotube goes). Or it can be ``cntfet_contact_rect`` to define a rectangular EB shape with a rectangular cavity. For all supported shape, see function ``ReadObjectInfo(...)`` in the file ``Source/Input/EmbeddedBoundaries.cpp``. Below we provide example of using ``cntfet_contact_cyl``. We assume that we are defining it for an object named Drain. - ``Drain.geom_type = cntfet_contact_cyl`` Here we define the geometry type for the Drain. - ``Drain.inner_radius=`` Here we specify the inner radius of the cavity. - ``Drain.thickness=`` Here we set the thickness of the cylinder. - ``Drain.center= Y-center> `` These are all float values that define the center of the cylinder in a domain. - ``Drain.height = `` This defines the height of the cylinder in the axial direction. This height protrudes the cylinder symmetrically around its center in the axial direction such that the height is as specified here. - ``Drain.direction=`` ```` can be 0, 1, or 2 for defining the axial direction as X, Y, or Z. - ``Drain.surf_soln=`` This parameter can be used to set the potential on the surface. Optionally we can set a parser for setting the potential as follows. - ``Drain.surf_soln_parser=`` Optionally we can use a parser for setting the potential, by setting this parameter 1. In this case, we do not define the previous parameter. This parameter is useful in case if the source potential varies with time. Note that in the case of steady state NEGF, we do not actually vary time but we use the time parameter ``t`` in the parser equation to vary the steps, e.g. to compute I-Vds characteristics by varying drain-source voltage (Vds) value. - ``Drain.surf_soln_function=`` Here we can write a string enclused parser expression as supported by AMReX. For example to vary drain-source voltage, we can write ```` as ``"SV + Vds_max - (Vds_max-Vds_min) * t"``, where ``SV`` is a constant voltage on the source (typically 0), ``Vds_max`` is the maximum drain-source voltage, ``Vds_min`` is the minimum drain-source voltage, and ``t`` is a `time` varying parameter from 0 to 1 to span the entire range from ``Vds_max`` to ``Vds_min``. Restart parameters ^^^^^^^^^^^^^^^^^^ - ``restart = 1`` set while restarting the code. - ``restart_step = `` Replace ```` with step to be restarted. Electrostatics module parameters ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ See AMReX documentation for MLMG parameters such as ``mlmg.set_verbose``, ``mlmg.max_order``, ``mlmg.absolute_tolerence``, and ``mlmg.relative_tolerance``. For debugging, we typically need: - ``mlmg.set_verbose = `` which sets the verbosity level for output of the MLMG multigrid solver. Useful for debugging. Usually set to 0. Diagnostic parameters ^^^^^^^^^^^^^^^^^^^^^ For the following parameters to work, set ``use_diagnostics=1``. - ``diag.specify_using_eb = `` With this flag, we can specify diagonstic regions using embedded boundaries. It is typically 1. - ``diag.objects = ... `` Here we can define multiple labels for different diagnostic objects. This label can be any string, e.g ``Z_rho``, which we will use next to output charge density on a plane. - ``Z_rho.geom_type = plane`` Here we are defining geometry type as a plane. - ``Z_rho.direction = `` can be 0, 1, or 2, to set X, Y, or a Z-plane. - ``Z_rho.location = `` This can be used to set the location of the Z-plane. - ``Z_rho.fields_to_plot = charge_density`` This plots the charge density on the plane. This field label should be defined in ``macroscopic.fields_to_define``. NEGF and Broyden module parameters ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - ``transport.NS_names = `` Optional parameter to define vector of string names for nanostructures. Folders with these names are created in the ``/negf/`` folder. - ``transport.NS_num = `` If ``transport.NS_names`` is not specified then we need to set this parameter defining the number of nanostructures. - ``transport.NS_type_default = CNT`` The default type of nanostructures. Here it is defined as CNT, referring to carbon nanotube. - ``transport.gather_field = phi`` Here ``phi`` refers to the name of the multifab defined to hold potential. - ``transport.deposit_field = charge_density`` Here ``charge_density`` refers to the named of the multifab defined to hold charge density values. - ``transport.NS_initial_deposit_value = `` This is the initial charge deposited on the surface of the material while starting the simulation. - ``transport.reset_with_previous_charge_distribution = `` Value of 1 means when the gate-source or drain-source voltages change (i.e. for step > 0) during sweeping while characterizing a FET, we initialize the charge on the material based on the converged charge at previous conditions. If set to 0, then the charge is always initialized with a value set for ``NS_initial_deposit_value``. At step 0, the charge is initialized with a value set for ``NS_initial_deposit_value`` regardless of what value we set for this parameter. - ``transport.selfconsistency_algorithm = broyden_second`` Currently only Broyden's modified second algorithm is supported in parallel. See the first ELEQTRONeX paper for details. - ``transport.Broyden_fraction = `` Value of the Broyden fraction is defined between 0 to 1. Typically 0.1. - ``transport.Broyden_norm_type = `` ```` can be relative or absolute. Typically we have used relative norm. - ``transport.Broyden_max_norm = `` Broyden iterations stop if the relative norm at all charge density sites is below this value. Typically set to 1.e-5. - ``transport.Broyden_threshold_maxstep = `` Broyden iterations stop if the number of iterations exceed the value defined here. Typically 100. The following parameters can be defined via ``NS_default`` prefix, with an option to override then by redefining it with a prefix defined by a nanostructure name, e.g. ``CNT`` for the example defined above. This is useful when we want to change the parameters only for specific nanostructures, while retaining the default behavior for the result of nanostructures. For example, if we want to define 10 nanotubes, all having most parameters common, but one or two parameters varying across nanotubes, then the common parameters can be defined with prefix ``NS_default``, while the nanostructure specific parameters can be defined with prefix, ``NS_1``, ``NS_2``, etc. Here we assume that we have defined ``transport.NS_names = NS_1 NS_2`` for these names to be valid. Below we use prefix ``NS_default``. - ``NS_default.num_unitcells = `` Number of unitcells of material. Note that for all examples with carbon nanotube, we have defined the CNT length in terms of number of unitcells. - ``NS_default.rotation_order = ...`` where ```` can be ``X``, ``Y``, or ``Z``. Material is initialized with its center at (0,0,0) coordinates. Rotation is applied to this material using Euler angles in the order specified here. ``NS_default.rotation_order = Z Y X`` would mean rotation is first applied about the Z axis, then Y, then X. - ``NS_default.rotation_angle_type = `` ```` can be ``D`` for degrees or ``R`` for radians. - ``NS_default.rotation_angles = `` where these values correspond to Euler angles. - ``NS_default.offset = `` The material geometry is centered around zero. This parameter lets up translate it. These values are floats. In multiple nanotube simulations, we have different offsets for each nanotube. - ``NS_default.contact_Fermi_level = `` This is the Fermi level defined as input in the time independent Green's function formalism. - ``NS_default.contact_mu_specified = `` 1 means that we provide the value of Fermi levels at the contacts. O means that it is computed based on the specified voltage on the embedded boundaries designated as source and drain. This parameter is set to 1, when we vary gate-source voltage, while keeping drain-source voltage fixed. It is set to 0, when we vary drain-source voltage, while keeping gate-source voltage fixed. - ``NS_default.contact_mu = `` If the above parameter is set to 1, then we define the value of contact Fermi levels here. E.g. ``Ef (Ef-Vds)`` where ``Ef`` is a constant defining the Fermi level of the source and while ``Ef-Vds`` is the Fermi level of the drain with ``Vds`` is a constant defining the drain-source bias. - ``NS_default.contact_T = `` These two values correspond to temperatures of source and drain contacts, respectively (in Kelvin). - ``NS_default.gate_terminal_type = `` where ```` can be an embedded boundary, in which case we need to define an embedded boundary for the gate, e.g. in the case of a gate-all-around geometry. Other option for ```` is ``Boundary``, in which case we need to define a Dirichlet with the name as ``Gate``, e.g. for a planar CNTFET geometry, we define ``boundary.low = neu(0.) neu(0.) dir(Gate)`` and ``boundary.Gate_function = "SV + Vgs_max - (Vgs_max-Vgs_min) * t"`` which defines a gate at the Zmin boundary. - ``NS_default.Fermi_tail_factors = `` These values are multipliers that define minimum and maximum tails for the Fermi function. Typically we define 14 for both values, meaning that Fermi function has tails ``-14kT`` and ``+14kT``. This parameter is used in the code while defining the integration region. - ``NS_default.eq_integration_pts = `` All 3 values are integers and they correspond to number of quadrature points used for integration using Gauss-Legendre polynomials. The three paths are used for integration the equilibrium region in the complex plane (see supplementary Fig.2 in the first ELEQTRONeX paper). Instead of integrating over a real line from Z_a to Z_b, we integrate along path 1: from Z_b to Z_c, path 2: from Z_c to Z_d, and path 3: from Z_d to Z_a. Note that the third path is curved. - ``NS_default.num_noneq_paths=`` In nonequilibrium, we have a second term to integrate, as defined in the Eq. (7) of the supplementary note 3 of the first ELEQTRONeX paper. This integration occurs over a line parallel to the real line (shifted by ``NS_default.E_zPlus_imag`` on the imaginary line). This path is by default assumed to be a single path of integration (```` is set to ``1``). However, it can be divided into ``3`` parts for the adaptive scheme. - ``NS_default.noneq_integration_pts=`` Here we define the number of integration points for each nonequilibrium path. If there is only a single path, then the value correspond to that single path. If there are 3 paths, then we need to provide 3 values. - ``NS_default.NS_default.num_recursive_parts=`` This number has to do with the recursive (serial) part of the calculation for calculating Green's function, in block tri-diagonal inversion algorithm (see explanation of supplementary Fig. 1(b) in the first ELEQTRONeX paper.)The integer value chosen for this parameter divides the recursive array into as many parts and overlaps recursive computation of each part with its copy from CPU to GPU. Ideally, we want to divide the array small enough that the time for the two tasks are equal. - ``NS_default.E_valence_min = `` This is the minimum value of the valence bandedge for integration (see supplementary Fig. 2). - ``NS_default.E_pole_max = `` This value is the parameter Delta in supplementary Fig. 2, which determines how many Fermi function poles we include on the imaginary plane in the contour integration. - ``NS_default.E_zPlus_imag = `` This is the small number eta in Eq.(2) of the first ELEQTRONeX paper. NEGF data output parameters ^^^^^^^^^^^^^^^^^^^^^^^^^^^ - ``NS_default.write_at_iter=`` 1 means we write out data (induced charge, potential, and relative/absolute norm) at each field site at each iteration. - ``NS_default.flag_compute_DOS=`` 1 means we compute and write the density of states of the material based on contact Fermi levels. - ``NS_default.flag_write_LDOS = `` This flag write out local density of states at each step (step refers to different Vgs or Vds values depending what we are varying). For this parameter to work, ``NS_default.flag_compute_DOS`` must be 1. - ``NS_default.flag_write_LDOS_iter = `` This flag writes out local density of states at each iteration at a given Vgs, Vds conditions. Useful for debugging. - ``NS_default.write_LDOS_iter_period=`` If the previous parameter is enabled, then this parameter can be used to set the interval with which the LDOS output is written out. - ``NS_default.flag_compute_flatband_dos=`` 1 means we compute the density of states of the material when the bands are flat, i.e. assuming that the contact Fermi levels are at 0. - ``NS_default.flatband_dos_integration_points=`` Number of integration points for the path defined by limits in the next parameter. - ``NS_default.flatband_dos_integration_limits= `` e.g. ``-1. 1.`` would mean we integrate from E=-1 eV to 1 eV. - ``NS_default.flag_write_integrand=`` 1 means we write out the integrands at the center of the channel. For example, in Fig. 3d of the first ELEQTRONeX paper, we plot integrands. - ``NS_default.write_integrand_interval=`` Without setting this parameter integrand is written out only at convergence. If we want to see integrand at specific Broyden iterations, we can specify the interval here. - ``NS_default.flag_write_charge_components=`` 1 means, we write out the individual parts of charge density that make up the induced charge density, e.g. the contribution from equilibrium path (part 1 of Eq. (7), computed using Residue theorem), part 2 of Eq. (7), computed by integrating over a real line, neutral charge density as defined in the paper. This is useful for debugging. Carbon nanotube parameters ^^^^^^^^^^^^^^^^^^^^^^^^^^ - ``CNT_default.type_id = m_index n_index`` These are integer values. Carbon nanotube is defined as (m_index, n_index) nanotube. - ``CNT_default.acc = bond_length`` bond_length is carbon-carbon bond length (0.142 nm). - ``CNT_default.gamma = 2.5 `` gamma is the overlap parameter in eV (see the first ELEQTRONeX paper). Point charge parameters ^^^^^^^^^^^^^^^^^^^^^^^^ - ``charge_density_source.type = point_charge`` This parameter chooses the external charge density source as point charges. - ``pc.default_charge_unit = `` This means default charge for each trap is ``e``. - ``pc.default_occupation = `` This means the default occupation for each trap is ````. 0 means completely deactivated, 1 means fully active. - ``pc.charge_density = `` This is the charge density in SI units. For 3D and 2D random distribution of charges the density is in units of e/nm^3 and e/nm^2, respectively. - ``pc.mixing_factor = `` This value refers to the simple mixing factor used for updating Vi in Eq.(5) of the second ELEQTRONeX paper. - ``pc.flag_vary_occupation = `` This flag is used to set traps with variable occupation according to Eq.(4) of the second ELEQTRONeX paper. - ``pc.Et = `` If the ``flag_vary_occupation`` flag is 1, then we can use this parameter to set Vt in the denominator in Eq.(4) of the second ELEQTRONeX paper on traps. Note that ``Et`` should have been defined as ``Vt``. - ``pc.V0 = `` This is the parameter Vo in Eq.(4). - ``pc.flag_random_positions = `` This flag determines whether we are going to initialize traps with a random distribution or specify them one by one. - ``pc.random_seed = `` This value corresponds to random seed used for generating trap distribution. - ``pc.offset = `` The trap locations are translated by this much amount in the X, Y, and Z directions when they are initialized with a random distribution (between 0 and 1). We can use this parameter to fix the minimum bounds of the cuboid region in which we want to initialize the traps. For example, we typically set this as ``pc.offset = (-GO_width/2. + 10*dx) (-G_length/2.) (ZOffset)`` where ``GO_width, G_length, Zoffset, dx`` are the constants used to set the gate-oxide width, gate length, Z value along the gate-oxide just above the vacuum/gate-oxide interface, and cell-size in the X direction, respectively. - ``pc.scaling = `` Similarly, the trap locations are scaled by these values when the traps are initialized with a random distribution (between 0 and 1). For example, we may apply scaling as ``pc.scaling = (GO_width - 20*dx) (G_length) (0)`` in addition to the offset defined above, which means that the traps will be introduced in the cuboid defined by the gate-oxide width 10dx short of the domain boundaries, entire gate length, and Z-plane marked by Zoffset. - ``pc.num = `` If we know the specific trap locations where we want to introduce traps, we can initialize this number. For this to work, we need to set ``pc.flag_random_positions=0``. - ``pc_.location = `` if ``pc.num`` is set then we can set the X, Y, Z locations of each ith trap with this parameter, where is a number from 1 to value set by pc.num. - ``pc.flag_write_individual_charge_files= `` As the name suggestions, with this parameter, we can output locations and other parameters corresponding to each trap at each step when their occupation values are converged.