Hierarchical approach to nanodesign

1

Hierarchical Approach to Nanodesign Viatcheslav Barkaline1, Yana Douhaya2, Aliaksandr Chashynski3, Alexander Pletezhov4 and Tomasz Szepieniec5 Abstract - NWChem package of quantum chemistry and molecular dynamics calculations was examined as a possible grid application in the frameworks of BalticGridII FP7 project. Realization of hierarchical approach to simulation of nanosystems on the basis of NWChem is discussed. NWChem installation and benchmark testing results on various architectures are described. Job submission from g-lite environment is discussed. Some preliminary results of carbon nanotubes array properties' calculations are presented. Keywords - Ab initio quantum chemistry, Molecular dynamics, NWChem package, Grid computing, g-Lite, JDL job, BalticGrid, Hierarchical Modeling, Nanodesign.

I. INTRODUCTION Current scientific and technological progress is usually understood in close links with the development of nanotechnology, the subject of which includes structures, processes and functions based on materials with properties defined on spatial scales 1 – 100 nm. Nanostructured substances may contain carbon nanotubes, fullerenes and fullerites, graphene sheets, dendrites, ceramics, zeolites, polymers and liquid crystals and metal and semiconductor nanostructures such as quantum dots, quantum walls, quantum wires. These components may be in various phase states or surface intermediate states on the solid-liquid, solid-gaseous, liquid-gaseous interfaces. Biological macromolecules such as lipids, proteins and DNA are studied now as an essential part of nanoelectronics and nanosensorics elementary basis. The fundamental difficulty of nanotechnology is the fact that nanostructures are so small that it is very hard to manipulate them precisely, and simultaneously too large for direct application of precise chemical methods such as genetic engineering for their treating. Methods of simulation of them have to be both fast and precise enough for the prediction and optimization of the electronic, atomic and phase structures, functional properties and chemical behaviour of nanomaterials 1

2

3

4

5

Viatcheslav Barkaline – System Dynamics & Material Mechanics lab., Belarusian National Technical University, 65 Nezavisimosti ave., Minsk, 220013, BELARUS, E-mail: [email protected]; Yana Douhaya - System Dynamics & Material Mechanics lab., Belarusian National Technical University, 65 Nezavisimosti ave., Minsk, 220013, BELARUS, E-mail: [email protected]; Aliaksandr Chashynski - System Dynamics & Material Mechanics lab., Belarusian National Technical University, 65 Nezavisimosti ave., Minsk, 220013, BELARUS, E-mail: [email protected]; Alexander Pletezhov – Intelligent Systems dept., Belarusian National Technical University, 65 Nezavisimosti ave., Minsk, 220013, BELARUS, E-mail: [email protected]. Tomasz Szepieniec - Academic Computer Center CYFRONET AGH, Nawojki str., 11, P.O.Box 386, 30-950, Cracow, POLAND, E -mail: [email protected]

and nanostructures under the external physical and chemical influences. Modeling of nanomaterials and nanosystems is a sophisticated task due to the fact that properties of nanomaterials are defined by interacted groups of hundreds of atoms and could hardly be computed by ab initio quantum mechanics (QM) methods even on supercomputers [1,2]. Long range atomic ordering, which is characteristic of crystals and reducing the number of system degrees of freedom is absent in nanomaterials while preserving here short range ordering, characteristic of liquids, does not allow to determine all necessary functional properties of nanomaterials. Components of nanosystems composed from such atomic groups contain millions of atoms and can be treated by molecular dynamics (MD) methods rather then QM, while the whole nanosystem can contain dozens of billions of atoms and only the finite elements’ (FE) continuous approximation can be applied to it. In general system theory it is established [3,4] that the description of such systems have to be decomposed into interrelated description levels when at every lower level only these properties and variables of the system are calculated which are necessary for modeling it on the upper level, and, on the other hand, the upper level determines modeling tasks and constrains for calculation on the lower level. The full model represents the hierarchy of the decision making according to Mesarovich [3]. Then the multilevel hierarchical system approach must be used in nanosystems computing too [5]. This approach generalizes multiscale simulation approach [6] and corresponds to combined traditional "up-to-down" and novel "down-to-up" technologies in nanodesign. At present paper the first results on the grid realization of hierarchical simulation system for nanodesign on the basis of NWChem QM and MD package in the framework of current Baltic Grid II FP7 project [7] are described and planned scientific work using the package is discussed.

II. SIMULATION LEVELS OF NANODESIGN The set of hierarchical levels proposed for modeling of nanostructured material properties and nanodesign are presented in Table 1. From a physical viewpoint modeling is based on exact ab initio quantum mechanical methods for clusters containing 10-100 atoms and representing all phases possible for given material. Such the number of atoms is enough to describe the processes of the creation and breaking of every chemical bond in the material. The description on this level is built on the notions of wave functions depending on the coordinates and spin variables of all electrons and atomic nuclei of the cluster and its time-independent Hamilton operator (isolated cluster) [8-9]. When nuclei can be considered as motionless the calculation-resource-saving electronic density functional theory can be applied [10]. The

MEMSTECH’2010, 20-23 April 2010, Polyana-Svalyava (Zakarpattya), UKRAINE

2 TABLE 1 HIERARCHICAL SIMULATION LEVELS OF NANOSYSTEMS NN

VI

Simulation level name. Basic equations

System level. State space models with initial conditions  x  Ax  Bu     y  Cx  Du

V

IV

III

Level of continuum multiphysics models. Maxwell’s and balance equations with state equations, boundary and initial conditions Mesoscopic level. Fluctuations’ dynamics on the basis of level V equations with fluctuation sources included Molecular dynamics level. Newton’s equations for all atoms  r

Space and time scales. Level elementary unit. Characteristic number of atoms.

Values calculated at the level

Comments

100 nm - 1 m, 1 ps - 1 year. System unit, subsystem, element of system with input u , output y and state x . 1023 atoms/mole

Long-term behavior of the system, transition and transfer functions, frequency responses, optimal control algorithms, regular and stochastic regimes features. Electromagnetic, mechanical and thermal behavior of microand nanosystems’ elements.

Models used in system theory and automatic control, theory of filtration, data mining

50 nm - 1 mcm, 50 ns -1 ms. Polycrystalline grains, granules, nanoclusters, powders. 106- 1010 atoms

Electromagnetic, mechanical and thermal behavior of microand nanosystems’ elements, linear response theory, fluctuation-dissipation relations.

1 - 500 nm, 1ps - 50 ns

viscosity, thermo- and electroconductivity, friction, elastic modules, piezomodules, dielectric constants of nonhomogeneous media, phase diagrams, state equations, steady state structures, phase transitions, non-equilibrium processes

Systems of stationary and time-dependent stochastic partial differential equations for fluctuated physical fields Solution of Newton equations for all atoms under thermostat influence

500 nm - 1 mm, 1 mcs -100 s. Continuum media element. 106- 1010 atoms

2

m

t

2

 F

Kinetics level. Liouville equation f t

II

Quantum statistical level. Von Neuman equation i

I

 Lf

ρ t

 H , ρ

Quantum mechanical level. Schrödinger equation ih

 t

1 nm - 10 mcm, 1 ps - 10 mcs

Multiple atomic clusters, macromolecul es, multiwall nanotubes. 103- 108 atoms

Systems of stationary and time-dependent partial differential equations for averaged physical fields

Probability distribution densities of various order

10 Å–100 nm, 100 fs – 10 ps Crystal unit cell, cluster in adjacent medium, single wall nanotubes. 10 – 1000 atoms

Intercluster interaction, cluster surfaces, stochastic energy spectrum

Statistic operator or density matrix ρ

0.1-20 Å, 1-1000 fs. Molecule, isolated cluster. 10 – 100 atoms

Interatomic interaction potentials, electronic subsystem distribution density

ab initio models, molecular orbitals’ theory, density function theory

 H

energy spectrum, stationary states, density of states, charge and spin densities for isolated cluster are determined as the function of nuclei coordinates and potential energy term for nuclei movements is calculated for a given electronic state. Introduction of such terms makes it possible to describe chemical reactions using quantum, quasi-classical and classical equations of motions. The models of quantum statistical level account for the environment of the given atomic cluster. Cluster is described by density matrix, arguments of which are related to both

cluster and its environment. Gibbs mixed ensemble formalism and conditioned probabilities technique for fixed environmental variables are used. Then, the thermodynamic description of the cluster is obtained. On the kinetic level the evolution of non-equilibrium system composed of hundreds of clusters in time-dependent external fields is studied on the basis of Liouville equation. As an alternative for the kinetic description the molecular dynamics models can be used in which classical equations of motion for all atoms are solved. On the basis of this approach it became

MEMSTECH’2010, 20-23 April 2010, Polyana-Svalyava (Zakarpattya), UKRAINE

3 possible to study systems containing up to millions of atoms on power enough computer systems. By this way the solids of 1-1000 nm3 volume can be described and nanomachines of molecular sizes can be simulated. For the description of larger systems it is necessary to average atomic properties over the volume of the upper level elementary unit (pseudoatom) and attribute to it thus obtained meanvalues. This is the mesoscopic description level. On this level not only meanvalues are essential but their fluctuations too [11]. Taking fluctuations into account is the main difference of mesoscopic level from macroscopic one on which it is possible to account for only meanvalues’ dynamics described as continuum medium equations. These equations are the balance equation for mass, energy, momentum, angular moment and entropy added by macroscopic Maxwell‘s equations for electromagnetic field and corresponding boundary conditions and state equations. Phenomenological parameters of the latter as well as the form of the equations, nevertheless, are obtained by mesoscopic level modeling. From the viewpoint of computer realization of hierarchy the results of mesoscopic modeling are used for the determination of the parameters of finite element method (FEM) based algorithms usually applied for continuum media and construction modeling. In some cases it is purposeful to use intermediate skeleton models in which chemical bonds are represented as rigid rods characterized by the parameters calculated from the force fields on molecular dynamics level. As the nanotechnology progresses the construction level goes down through the scale hierarchy to quantum level, the elementary units of every level can be not only material clusters but also constructions.

controllable selectivity, which could be refined by introducing additional electrical output, for which changes of electrical impedance due to the changes of dielectric permittivity and electrical conductivity lead to the change of electric current through CNT array medium. Acoustic and electric channels interact due to the changes of adsorption and diffusion conditions of molecules adsorbed on CNT with applied voltage in electric channel. This effect makes it possible to realize time resolution of adsorption control and control of sorption kinetics. The selectivity of the sensor can be increased by chemical modification of CNT. As an example of such modification at the down right part of Fig. 1 the bovine pancreatic trypsin inhibitor adsorbed on (10,10) carbon nanotube array is presented in real proportion.

III. CARBON NANOTUBE ARRAY BASED CHEMICAL SENSOR WITH ACOUSTIC PICKUP - AN EXAMPLE OF

Both electronic noses (adsorption from gaseous atmosphere) and electronic tongues (adsorption from liquid phase) can be realized. Design of such sensors means study of their functioning on several levels of description: Quantum mechanical study of the adsorption of gases on carbon nanotubes. By DFT calculation of hydrogen and oxygen single molecules adsorption on carbon nanotube surface with cc-pvdz basis set we have shown that adsorption has physical nature without creation of chemical bonds of these gases molecules with CNT (Fig. 2, 3). This means that the study of gas molecules distribution in the array pores can be studied by molecular dynamics methods. Molecular dynamics study of distribution of adsorbed molecules in CNT array pores. This distribution and adsorption capacity of array depends on intertube distance a (Fig. 4). When a< 3.5 Å, the oxygen molecules build linear chains along nanotubes at the centers of the array pores, while for larger a oxygen layers are created along outer surfaces of nanotubes. As oxygen content rises van der Waals minima for oxygen molecules become deeper and corresponding intertube distances rise too - array “swells”. For the case of linear oxygen chains it is shown in Fig. 5. Maximal oxygen content for this case is nearly 320 carbon atoms per one oxygen molecule. To predict change of acoustic properties of CNT array due to the gas adsorption it is necessary to know the dependence of macroscopic characteristics of the array, i.e. its

HIERARCHICAL TASK

To clarify the hierarchical nature of nanodesign tasks let us briefly discuss the simulation of concrete nanodevice – chemical gas sensor with sensing layer composed of ordered carbon nanotube (CNT) arrays, which seem to be one of the most promising achievements of current nanotechnology, and surface acoustic wave (SAW) pickup [12]. Such arrays may consist of single wall and multiwall nanotubes with diameter from dozens to hundreds angstroms and lengths up to several micrometers and present an example of highly ordered dispersive medium with significant contribution of van der Waals interactions to all thermodynamic properties of the material. SAW chemical sensors are based on the effects of adsorbed molecules on geometrical, elastic and electric properties of the gas-sensing layer and corresponding mass-loading of the working surface of substrate carrying SAW (central part of Fig. 1). These effects lead to SAW phase velocity local changes (modulation in the acoustic channel) determining sensor output signal (changes of resonant frequency for SAW resonators and time delay for delay lines, down left part of Fig. 1). The most promising features of nanotube array based SAW chemical sensor is broad dynamical range and high

Fig.1 CNT-array based surface acoustic wave nanosensor structures

MEMSTECH’2010, 20-23 April 2010, Polyana-Svalyava (Zakarpattya), UKRAINE

4 mass density and elastic moduli, from the adsorption value. Elastic moduli C IJ can be calculated from the dependences of microscopic van der Waals energy of the array E vdW from the

a)

macroscopic density [12]. The equation (1) manifests the interrelation of microscopic and macroscopic levels of description and corresponds to effective media approach [13].

b) a) b) c) Fig. 4 Basic shapes of oxygen filling of (10,10) CNT array pores: a) intertube distance а> $NAME.nw export NWCHEM_RUN=$VO_GAUSSIAN_SW_DIR/NWChem5.1/nwchem.x $NWCHEM_RUN $NAME.nw > $NAME.log 2>&1 tar cvf $NAME.result.tar $NAME* gzip -9 $NAME.result.tar echo "--- All Done ---"

and siosi6.tar.gz is archive, containing siosi6.nw NWChem input file. The command for job submitting was glite-wms-job-submit -a siosi6.jdl

The result of benchmark calculations shows that some optimization of grid cluster architecture is needed to realize the full power of NWChem code.

V. PROTOTYPE OF HIERARCHICAL SIMULATION SYSTEM FOR NANODESIGN

The prototype of hierarchical simulation system for nanodesign is under realization on SKIF K-1000 cluster of

FEM Simulation of constructions (ANSYS, LSDYNA, ELMER, OpenFOAM, COMSOL)

FE USERS

Assembler of finite element properties from nanoelements’ properties

NA

System level software (MATLAB+ SIMULINK)

SY Generator of nanoelements of the construction

NG

NWChem, NAMD calculation of nanoelements properties

QM/MD Fig.9 NWChem integration with FEM packages to nanodesign hierarchical simulation system. Dashed lines correspond to the connections under development

Nanodesign begins at SY level where the input-output characteristics of nanodevice are defined on the basis of its system theory model and characteristics of its elements such as voltage-current characteristics or frequency response curves. If all elements and their connections are standard, the task of nanodesign is solved. If some elements are non standard, their FE models are built. If all material properties of such models are known and nanoelements are absent in the structure of such elements, the task of nanodesign is solved by choosing corresponding FE models and storing them in the library of standard elements. If material property of some finite element is unknown, its atomic model is generated by NG (Fig.10) and the property is calculated on MD or, if necessary, QM levels. NWChem package was chosen for these levels due to the implementation of combined QM/MD method into it, when part of the system is treated by QM while its environment - by MD methods. Calculated property is stored in the library of material properties. If some finite element has nanostructure and composed of nanoelements, their atomic models are built. Then the structure of element is assembled by NA from nanoelements in accordance with known or proposed composition and the properties of the element are calculated by MD simulation or material science approaches, when the property of the element is represented as a function of corresponding properties of nanoelements, their volume content in the element and some

MEMSTECH’2010, 20-23 April 2010, Polyana-Svalyava (Zakarpattya), UKRAINE

9

QM

Fig.10 Various atomic representation of the same crystalline finite element by nanogenerator NG. Coordinates of the element vertexes and its crystallographic orientation are input and atomic coordinates are output of NG. Every finite element with unknown properties is represented as the atomic system with the same geometry, for which these properties can be calculated

other optimization parameters. After such a calculation one returns to FE level or, for device composed from nanoelements only, to SY. Grid realization of hierarchical simulation system for nanodesign is planned on the basis of a directed acyclic graph (DAG) JDL job type [18]. DAG represents a set of jobs where the input, output, or execution of one or more jobs depends on one or more other jobs. The jobs are nodes (vertices) in the correspondent graph and the edges (arcs) identify the dependencies between jobs. Although such job type does not require a great number of new attributes with respect to ordinary JDL job, the structure of the JDL description for a DAG differs significantly from the one for a job and it is for sure more complex. It is important to note that upon submission of a DAG, besides the identifier associated with each node, the Workload Management System (WMS) assigns also to the DAG itself an identifier that has to be used as the handle for monitoring and controlling the whole DAG. Proposed general DAG structure of hierarchical nanodesign JDL job is presented in Fig.11. Unidirectional arcs connecting nodes means that finish node can be launched only if start node is successfully ended. It is necessary to stress that any subgraph containing SY node corresponds to possible nanodesign job too. Due to the complexity of the interrelations between jobs included into DAG it seems purposeful to develop special user interface for their preparing.

VI. PROSPECTS NWChem package was defined as a pilot application for gridification in the frameworks of BalticGrid-II FP7 project. At present there are four types of NWChem working installations throughout this project: • • • •

at CYFRONET cluster in Cracow with MPI (gaussian VO); at SKIF K-1000 cluster (288 nodes) of UIIP under Fedora 8 x86_64, with OpenMPI (not yet in Grid); at small NICH BNTU cluster (16 cores) under SL 4.6, with MPICH (not yet in Grid); various single node versions for PC. Other parts of the hierarchical simulation system for

MD

NG

QM/MD

NA

FE

SY Fig.10 General DAG structure of JDL job for hierarchical nanodesign task

nanodesign will be installed and tested throughout BalticGridII consortium and will be available through g-lite environment. Optimization of Grid installations of NWChem will be undertaken to get maximal productivity. Special software for preparing NWChem job and DAG JDL job for nanodesign will be developed. First-priority scientific tasks dealing with NWChem and hierarchical nanodesign are defined: • MD calculation of vibration spectra of carbon nanotube arrays for sensor and UHF technique applications; • DFT calculation of the adsorption properties of carbon nanotubes and their bundles; • modeling of interaction of DNA and other biopolymers with carbon nanotubes for biosensorics; • pseudopotential calculation of electronic properties of the fullerene(ferrocene)2 single crystal as bioactive nanoclusters; • QM/MD calculation of Fe, Co and Ni clusters intercalated into fullerenes and carbon nanotubes and their influence on adsorption of simple gases and magnetic properties of carbon nanostructured materials. • hierarchical optimization of the structures of nanosensors and UHF nanodevices on carbon nanotube arrays. Due to the growing interest to nanodesign problems among scientists and developers it seems purposeful to arise the question of launching corresponding special interest groups and virtual organizations.

VII. ACKNOWLEDGEMENTS Work was supported by EC FP7 project BalticGrid-II, grant 223807, Belarusian state program “Nanotech 2006-2010”, grant 4.19, and the “Triada” Scientific Technical Program of the Belarus-Russia Union State, grant ПА 2.7. Authors want to express gratitude to EMSL for providing them with NWChem and ECCE packages.

MEMSTECH’2010, 20-23 April 2010, Polyana-Svalyava (Zakarpattya), UKRAINE

10

REFERENCES [1] H. Primas, Chemistry, Quantum mechanics and Reductionism. Berlin, Heidelberg, New York: SpringerVerlag, 1981. [2] W. Kohn, “Electronic structure of matter – wave functions and density functionals,” Physics-Uspekhi, vol. 172, N3, pp.336-348, 2002. [3] М. Mesarovich, D.Mako, Y. Takahara, Theory of hierarchical multilevel systems. New York and London : Academic press, 1970. [4] S. Novikava, K. Miatluk, S. Ganchrova, A. Ivahow, A. Zhybul, A. Danichaw, P. Buka, V. Siargeichyk, A. Demyanenka, “Theory in Hierarchical Knowledge Networks,” Studies in Informatics and Control, vol.6, N 1, pp.75-85, 1997. [5] V.V.Barkaline, “Quantum levels of the hierarchy of nanomaterials models”, in Modern methods of machine design, Issue 2, Vol.2. Quality of machine building products, material and construction design, pp.88-93. Minsk: «Technoprint» Publishing, 2004. [6] T. Cagin, J. Che, Y. Qi, Y. Zhou, E. Demiralp, G. Gao, W.A. Goddard III, “Computational materials chemistry at the nanoscale”, Journal of Nanoparticle Research, vol.1, pp.51-69, 1999. [7] http://www.balticgrid.org. [8] S. Wilson, Electron correlation in molecules. Oxford: Clarendon Press, 1984. [9] B.K. Novosadov, Methods of solving quantum chemistry equations. Мoscow: Nauka, 1988. [10] R.M. Dreizler, E.K.U. Gross, Density Functional Theory. An Approach to Quantum Many-Body Problem. Berlin, Heidelberg, New York, London, Paris, Tokio, Hong Kong, Barcelona: Springer-Verlag, 1990. [11] V.F. Gantmakher, M.V. Feigel'man, “Mesoscopic unification,” Physics – Uspekhi, vol. 4, N2, pp.105-108, 1998. [12] V.V.Barkaline, A.S. Chashynski, P.А. Zhuchek, “Acoustic properties of carbon nanotube arrays as chemical sensor elements,” Reviews on Advanced Materials Science, vol. 20, N 1, pp.28-36, 2009. [13] G.M. Odegard, T.S. Gates, L.M. Nicholson, K.E.Wise, “Equivalent-Continuum Modeling with Application to Carbon Nanotubes.” NASA/TM-2002-211454, 2002. [14] L.M.Dorozhkin, I.A. Rozanov, “Acoustic Wave Chemical Sensors for Gases,” Journal of Analytical Chemistry, vol.56, No. 5, pp.399–416, 2001. [15] E.J. Bylaska, W.A. de Jong, N. Govind, K. Kowalski, T. P. Straatsma, M. Valiev, D. Wang, E. Apra, T.L. Windus, J. Hammond, P. Nichols, S. Hirata, M.T. Hackler, Y. Zhao, P.-D. Fan, R.J. Harrison, M. Dupuis, D.M.A. Smith, J. Nieplocha, V. Tipparaju, M.Krishnan, Q. Wu, T. van Voorhis, A. A. Auer, M. Nooijen, E. Brown, G. Cisneros, G.I. Fann, H. Fruchtl, J. Garza, K. Hirao, R. Kendall, J.A. Nichols, K. Tsemekhman, K. Wolinski, J. Anchell, D. Bernholdt, P. Borowski, T.Clark, D. Clerc, H. Dachsel, M. Deegan, K.Dyall, D. Elwood, E. Glendening, M. Gutowski, A. Hess, J. Jaffe,

B. Johnson, J. Ju, R. Kobayashi, R. Kutteh, Z. Lin, R.Littlefield, X. Long, B. Meng, T. Nakajima, S. Niu, L. Pollack, M. Rosing, G. Sandrone, M. Stave, H. Taylor, G. Thomas, J. van Lenthe, A. Wong, Z. Zhang, NWChem, A Computational Chemistry Package for Parallel Computers, Version 5.1. Richland, Washington 99352-0999: Pacific Northwest National Laboratory, 2007. [16] http://www.emsl.pnl.gov/docs/nwchem/nwchem.html [17] http://ecce.emsl.pnl.gov/ [18] Job Description Language Attributes Specification for the gLite Middleware (submission through WMProxy Service). Document identifier: EGEE-JRA1-TEC590869-JDL-Attributes-v0-9.doc, Document link: https://edms.cern.ch/document/590869/1.

VIII. CONCLUSION In this paper the physical and computational aspects of hierarchical approach to nanodesign are presented and discussed. Prospects of NWChem package as main tool for quantum mechanical simulation of nanostructures are estimated.

MEMSTECH’2010, 20-23 April 2010, Polyana-Svalyava (Zakarpattya), UKRAINE